Oct 30 21:08:50.213682 ip-10-0-132-134 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 21:08:50.213694 ip-10-0-132-134 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 21:08:50.213702 ip-10-0-132-134 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 21:08:50.213982 ip-10-0-132-134 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 21:09:00.304660 ip-10-0-132-134 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 21:09:00.304681 ip-10-0-132-134 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2c2d3cb305324cb2b20f679ae91aaa6d --
Oct 30 21:11:05.436089 ip-10-0-132-134 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:11:05.785186 ip-10-0-132-134 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:11:05.785186 ip-10-0-132-134 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:11:05.785186 ip-10-0-132-134 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:11:05.785186 ip-10-0-132-134 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:11:05.788234 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788152    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:11:05.788234 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788235    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788240    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788244    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788247    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788250    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788253    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788256    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788258    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788261    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788277    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788281    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788284    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788287    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788289    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788292    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788294    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788297    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788299    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788302    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:11:05.788297 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788305    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788309    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788313    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788316    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788319    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788321    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788323    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788326    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788328    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788331    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788334    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788336    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788339    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788342    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788344    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788347    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788350    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788352    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788354    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:11:05.788750 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788357    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788359    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788361    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788364    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788367    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788369    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788372    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788374    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788376    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788380    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788382    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788385    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788388    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788390    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788393    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788395    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788398    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788401    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788403    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788405    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:11:05.789227 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788408    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788410    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788412    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788414    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788417    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788419    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788421    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788424    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788426    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788429    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788431    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788435    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788437    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788439    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788442    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788444    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788446    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788449    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788452    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:11:05.789723 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788454    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788456    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788459    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788462    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788464    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788466    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788469    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788471    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788474    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788476    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788479    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788481    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788484    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788486    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788488    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788491    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788493    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788496    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788499    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788501    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:11:05.790187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788503    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788507    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788511    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788514    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788517    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788519    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788523    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.788525    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789325    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789332    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789335    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789338    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789343    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789347    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789350    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789353    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789356    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789359    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789361    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:11:05.790664 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789364    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789368    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789372    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789376    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789379    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789382    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789384    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789387    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789390    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789393    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789395    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789398    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789401    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789403    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789406    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789408    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789411    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789413    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789416    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789418    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:11:05.791116 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789421    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789423    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789425    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789428    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789430    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789436    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789439    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789442    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789445    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789448    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789450    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789453    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789456    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789460    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789462    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789465    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789467    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789470    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789473    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:11:05.791576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789475    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789477    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789480    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789482    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789485    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789488    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789490    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789493    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789495    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789498    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789500    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789503    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789505    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789508    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789510    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789513    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789515    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789518    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789520    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789523    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:11:05.792017 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789527    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789529    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789532    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789535    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789537    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789540    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789543    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789545    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789548    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789550    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789553    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789556    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789558    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789561    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789563    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789566    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789568    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789571    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789573    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789576    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:11:05.792505 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789578    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789580    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789583    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789585    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789588    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789590    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789593    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789595    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789599    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789601    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789604    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789606    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789609    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789612    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789614    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.789616    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790140    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790146    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790149    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:11:05.793087 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790153    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790155    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790158    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790161    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790164    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790166    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790169    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790171    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790174    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790176    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790180    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790184    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790187    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790190    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790193    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790196    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790199    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790202    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790204    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790206    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:11:05.793569 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790209    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790212    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790214    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790217    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790219    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790222    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790225    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790227    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790229    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790232    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790235    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790250    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790253    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790255    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790259    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790262    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790264    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790267    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790270    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790272    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:11:05.794068 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790274    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790276    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790279    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790281    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790284    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790286    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790288    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790291    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790293    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790296    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790298    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790301    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790303    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790306    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790308    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790310    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790313    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790315    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790318    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790321    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:11:05.794558 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790323    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790325    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790328    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790330    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790333    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790335    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790338    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790340    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790342    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790345    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790347    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790350    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790352    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790355    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790357    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790360    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790362    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790364    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790367    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790369    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:11:05.795028 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790371    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790374    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790376    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790378    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790381    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790383    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790385    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790387    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790390    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790392    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790394    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790397    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790400    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790402    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790405    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790407    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790410    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790412    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790414    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:11:05.795519 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790429    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790433    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790440    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.790443    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791181    2574 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791191    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791198    2574 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791203    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791207    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791210    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791215    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791219    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791222    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791225    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791229    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791232    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791235    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791238    2574 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791241    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791244    2574 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791246    2574 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791249    2574 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791253    2574 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:11:05.795971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791256    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791259    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791262    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791266    2574 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791270    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791274    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791277    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791280    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791283    2574 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791286    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791289    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791294    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791297    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791301    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791304    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791307    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791310    2574 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791313    2574 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791316    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791320    2574 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791323    2574 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791326    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791329    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791332    2574 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:11:05.796486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791336    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791339    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791342    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791345    2574 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791348    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791351    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791354    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791357    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791360    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791362    2574 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791365    2574 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791371    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791375    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791378    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791381    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791384    2574 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791387    2574 flags.go:64] FLAG: --help="false"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791390    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791393    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791396    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791399    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791402    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791406    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:11:05.797111 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791409    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791411    2574 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791414    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791417    2574 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791419    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791423    2574 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791425    2574 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791428    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791431    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791434    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791437    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791439    2574 flags.go:64] FLAG: --lock-file=""
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791442    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791445    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791448    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791454    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791456    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791459    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791462    2574 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791465    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791468    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791473    2574 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791476    2574 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791480    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791483    2574 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:11:05.797706 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791487    2574 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791490    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791493    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791496    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791499    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791502    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791505    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791508    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791515    2574 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791518    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791521    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791524    2574 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791526    2574 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791529    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791532    2574 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791535    2574 flags.go:64] FLAG: --port="10250"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791538    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791541    2574 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0c6b1289c15dcabf0"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791544    2574 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791547    2574 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791549    2574 flags.go:64] FLAG: --register-node="true"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791552    2574 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791556    2574 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791559    2574 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791562    2574 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791565    2574 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:11:05.798632 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791568    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791571    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791575    2574 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791579    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791582    2574 flags.go:64] FLAG: --runonce="false"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791585    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791588    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791591    2574 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791594    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791597    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791601    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791604    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791606    2574 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791609    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791612    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791615    2574 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791618    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791620    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791623    2574 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791626    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791632    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791635    2574 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791637    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791640    2574 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791643    2574 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:11:05.799408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791646    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791648    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791651    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791654    2574 flags.go:64] FLAG: --v="2"
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791659    2574 flags.go:64] FLAG: --version="false"
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791663    2574 flags.go:64] FLAG: --vmodule=""
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791667    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.791670    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791841    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791846    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791849    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791852    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791855    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791858    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791861    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791863    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791866    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791869    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791872    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791874    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791876    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:11:05.800008 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791879    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791882    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791884    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791886    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791889    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791891    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791893    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791896    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791898    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791901    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791903    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791905    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791908    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791910    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791913    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791915    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791919    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791923    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791926    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:11:05.800485 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791928    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791932    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791934    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791937    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791943    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791946    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791948    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791951    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791955    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791957    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791959    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791962    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791965    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791967    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791970    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791972    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791975    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791977    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791980    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791982    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:11:05.800939 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791985    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791987    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791989    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791992    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791994    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791997    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.791999    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792001    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792004    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792006    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792009    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792011    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792013    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792017    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792020    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792023    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792025    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792029    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792031    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792034    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792037    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:11:05.801464 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792039    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792042    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792044    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792047    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792049    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792051    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792054    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792056    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792058    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792061    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792063    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792065    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792069    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792072    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792075    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792077    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792080    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792082    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792084    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792087    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:11:05.801989 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792089    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792091    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792094    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792096    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792100    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792102    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792105    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792107    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792109    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792112    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792115    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792117    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792119    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.792145    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792187    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:11:05.802472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792190    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792193    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792195    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792198    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792200    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792202    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792205    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792207    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792209    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792212    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792214    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792217    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792219    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792221    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792224    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792226    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792228    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792231    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792233    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:11:05.802853 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792236    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792238    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792243    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792245    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792247    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792250    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792252    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792254    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792257    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792259    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792262    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792265    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792267    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792270    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792272    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792274    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792277    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792279    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792282    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:11:05.803273 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792285    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792289    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792292    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792295    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792298    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792300    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792302    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792305    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792307    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792309    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792312    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792314    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792316    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792319    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792323    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792326    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792329    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792331    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792333    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792336    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:11:05.803699 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792338    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792341    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792343    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792346    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792348    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792351    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792353    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792355    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792358    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792360    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792362    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792365    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792367    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792369    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792371    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792374    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792376    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792379    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792381    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792384    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:11:05.804176 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792386    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792388    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792391    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792393    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792395    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792398    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792400    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792404    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792406    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792408    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792411    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792413    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792427    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792430    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792433    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792437    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792441    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792444    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792446    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792449    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:11:05.804630 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792451    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792453    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792457    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792459    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792461    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792464    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.792466    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.800434    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.800451    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.800469    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.800474    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800533    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800538    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800542    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800545    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800549    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800551    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800555    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800557    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800560    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:11:05.805135 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800562    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800565    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800567    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800570    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800572    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800574    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800577    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800579    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800582    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800584    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800587    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800589    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800592    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800594    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800596    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800599    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800603    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800605    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800609    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800613    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:11:05.805637 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800617    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800620    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800623    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800625    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800628    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800631    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800633    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800635    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800638    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800640    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800643    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800646    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800648    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800651    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800653    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800656    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800658    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800661    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800664    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:11:05.806145 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800666    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800669    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800671    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800673    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800676    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800678    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800680    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800683    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800686    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800688    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800691    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800693    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800696    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800698    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800702    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800706    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800709    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800712    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800714    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:11:05.806600 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800717    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800719    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800722    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800724    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800727    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800729    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800731    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800734    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800736    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800739    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800741    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800744    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800748    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800750    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800753    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800755    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800758    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800760    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800762    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800765    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:11:05.807085 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800767    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800769    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800772    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800775    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800777    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800780    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800783    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800785    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800788    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800806    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800809    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800812    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800815    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800817    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800819    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800822    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800824    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800826    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:11:05.807590 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.800829    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.800856    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801006    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801011    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801014    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801017    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801020    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801023    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801026    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801029    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801031    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801035    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801037    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801040    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801042    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801045    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:11:05.808049 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801047    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801051    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801053    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801055    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801058    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801060    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801063    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801066    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801068    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801070    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801073    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801075    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801078    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801080    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801082    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801085    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801087    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801090    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801092    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:11:05.808428 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801095    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801097    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801099    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801102    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801105    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801116    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801119    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801121    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801124    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801126    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801129    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801131    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801134    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801137    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801141    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801162    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801165    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801168    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801171    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:11:05.809007 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801173    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801176    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801179    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801181    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801184    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801187    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801189    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801192    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801194    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801197    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801200    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801202    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801205    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801207    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801209    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801213    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801217    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801220    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801223    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:11:05.809473 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801226    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801229    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801231    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801234    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801236    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801239    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801241    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801243    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801246    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801249    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801251    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801254    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801256    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801259    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801261    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801263    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801266    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801268    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801271    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801273    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:11:05.810222 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801275    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801278    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801280    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801283    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801285    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801288    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801290    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801293    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801295    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801298    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801300    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801302    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801305    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801308    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801311    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.801334    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:11:05.810780 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801364    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801369    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801372    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801374    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801377    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801380    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801382    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801385    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801387    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801390    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801392    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801395    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801398    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801400    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801403    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801405    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801407    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801410    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801412    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801414    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:11:05.811187 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801417    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801422    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801424    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801427    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801430    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801432    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801434    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801437    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801439    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801442    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801445    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801448    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801450    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801453    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801456    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801459    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801462    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801464    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801467    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801470    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:11:05.811681 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801472    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801475    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801477    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801480    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801482    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801485    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801487    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801490    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801492    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801495    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801497    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801500    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801502    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801505    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801508    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801510    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801513    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801515    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801517    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:11:05.812286 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801520    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801522    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801525    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801527    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801530    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801533    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801535    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801539    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801543    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801545    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801547    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801550    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801552    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801555    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801559    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801562    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801565    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801568    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801570    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:11:05.812743 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801573    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801575    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801578    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801580    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801583    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801585    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801587    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801590    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801592    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801595    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801597    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801600    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801602    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801604    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801607    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801609    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801613    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801616    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801620    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801623    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:11:05.813243 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801625    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801629    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801632    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801634    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801637    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801639    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801641    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:05.801644    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.801705    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.803464    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.804286    2574 server.go:1014] "Starting client certificate rotation"
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.804381    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.804408    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.806099    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:11:05.813749 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.807517    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:11:05.818334 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.818316    2574 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:11:05.821268 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.821255    2574 log.go:25] "Validated CRI v1 image API"
Oct 30 21:11:05.822035 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.822013    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:11:05.825112 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.825081    2574 fs.go:135] Filesystem UUIDs: map[07be4522-0d4c-4744-8a0f-79c744ca9085:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a44f5419-2dab-4964-8cc8-c35488adeee4:/dev/nvme0n1p4]
Oct 30 21:11:05.825207 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.825106    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:11:05.832271 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.832122    2574 manager.go:217] Machine: {Timestamp:2026-10-30 21:11:05.830977072 +0000 UTC m=+0.291173706 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3206461 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2627451245dedc35810aaceb3d0358 SystemUUID:ec262745-1245-dedc-3581-0aaceb3d0358 BootID:2c2d3cb3-0532-4cb2-b20f-679ae91aaa6d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:41:0b:b5:7b:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:41:0b:b5:7b:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:d7:5c:97:90:7f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:11:05.832271 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.832263    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:11:05.832408 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.832396    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:11:05.833915 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.833879    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:11:05.834105 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.833917    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-134.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:11:05.834149 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.834108    2574 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:11:05.834149 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.834116    2574 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:11:05.834202 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.834155    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:11:05.834202 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.834174    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:11:05.834862 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.834852    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:11:05.837104 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.837092    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:11:05.837240 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.837230    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:11:05.837827 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.837817    2574 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:11:05.837870 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.837831    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:11:05.837870 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.837843    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:11:05.837870 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.837852    2574 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:11:05.837870 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.837860    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:11:05.838204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.838191    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:11:05.840247 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.840225    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:11:05.844741 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.844726    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:11:05.845404 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.845390    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:11:05.845460 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.845422    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:11:05.845586 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.845574    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:11:05.845615 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.845594    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:11:05.845615 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.845603    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:11:05.845615 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.845612    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:11:05.845688 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.845625    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:11:05.845688 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.845640    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:11:05.846051 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.846041    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:11:05.846088 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.846052    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:11:05.846088 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.846060    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:11:05.846088 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.846066    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:11:05.846088 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.846075    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:11:05.846088 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.846085    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:11:05.846937 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.846928    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:11:05.846937 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.846938    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:11:05.850774 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.850762    2574 server.go:1263] "Started kubelet"
Oct 30 21:11:05.850882 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.850828    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:11:05.850959 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.850912    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:11:05.851005 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.850978    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:11:05.851544 ip-10-0-132-134 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:11:05.851947 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.851929    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:11:05.854106 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.854088    2574 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:11:05.856292 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:05.856261    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:11:05.856429 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.856415    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:11:05.857959 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.857946    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:11:05.858146 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:05.858114    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-134.us-west-1.compute.internal\" not found"
Oct 30 21:11:05.858462 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.858441    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:11:05.858462 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.858443    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:11:05.858573 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.858469    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:11:05.858603 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.858577    2574 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:11:05.858603 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.858586    2574 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:11:05.859078 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.859063    2574 factory.go:55] Registering systemd factory
Oct 30 21:11:05.859648 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.859637    2574 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:11:05.859862 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.859848    2574 factory.go:153] Registering CRI-O factory
Oct 30 21:11:05.859897 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.859866    2574 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:11:05.859934 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.859920    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:11:05.859965 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.859939    2574 factory.go:103] Registering Raw factory
Oct 30 21:11:05.859965 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.859952    2574 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:11:05.860753 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.860734    2574 manager.go:319] Starting recovery of all containers
Oct 30 21:11:05.867551 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.867499    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:11:05.873697 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.873679    2574 manager.go:324] Recovery completed
Oct 30 21:11:05.877846 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.877832    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:11:05.879493 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.879471    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:11:05.879557 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.879505    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:11:05.879557 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.879518    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:11:05.880026 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.880003    2574 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:11:05.880026 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.880017    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:11:05.880131 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.880038    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:11:05.880131 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.880048    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:11:05.880751 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.880738    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:11:05.880835 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.880762    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:11:05.880835 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.880775    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:11:05.881020 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.880999    2574 policy_none.go:50] "Start"
Oct 30 21:11:05.881020 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.881016    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:11:05.881207 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.881030    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:11:05.881974 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.881963    2574 policy_none.go:44] "Start"
Oct 30 21:11:05.919415 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.919382    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:11:05.922019 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.922004    2574 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:11:05.922101 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:05.922039    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:11:05.922101 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.922048    2574 server.go:87] "Starting device plugin registration server"
Oct 30 21:11:05.922330 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.922318    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:11:05.922364 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.922332    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:11:05.922437 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.922423    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:11:05.922516 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.922505    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:11:05.922566 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.922517    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:11:05.923043 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:05.923023    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:11:05.923105 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:05.923077    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-134.us-west-1.compute.internal\" not found"
Oct 30 21:11:05.981377 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.981351    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:11:05.981529 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.981383    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:11:05.981529 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.981406    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:11:05.981529 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:05.981445    2574 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:11:05.991873 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:05.991845    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-134.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:11:05.991873 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:05.991858    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-134.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:11:05.992730 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:05.991868    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-134.us-west-1.compute.internal.18e36af26019eccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-134.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-134.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:11:05.850531021 +0000 UTC m=+0.310727654,LastTimestamp:2026-10-30 21:11:05.850531021 +0000 UTC m=+0.310727654,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-134.us-west-1.compute.internal,}"
Oct 30 21:11:06.023295 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.023274    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:11:06.024112 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.024095    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:11:06.024214 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.024127    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:11:06.024214 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.024143    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:11:06.024214 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.024173    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.078313 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.078212    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:11:06.078313 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.078255    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dnwld"
Oct 30 21:11:06.082638 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.082485    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"]
Oct 30 21:11:06.083070 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.083058    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:11:06.085678 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.085661    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:11:06.085766 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.085690    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:11:06.085766 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.085702    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:11:06.086905 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.086893    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:11:06.087059 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.087042    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.087112 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.087074    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:11:06.087635 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.087617    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:11:06.087703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.087650    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:11:06.087703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.087664    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:11:06.087703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.087682    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:11:06.087867 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.087707    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:11:06.087867 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.087722    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:11:06.088854 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.088837    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.088949 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.088869    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:11:06.089709 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.089686    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:11:06.089709 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.089711    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:11:06.089837 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.089726    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:11:06.110056 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:06.110026    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-134.us-west-1.compute.internal\" not found" node="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.113209 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:06.113191    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-134.us-west-1.compute.internal\" not found" node="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.147214 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.147192    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:11:06.148518 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.148503    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:11:06.159644 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.159615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.159737 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.159652    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d2a11b2240e0ab276a762b4e71b7c25e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"d2a11b2240e0ab276a762b4e71b7c25e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.159737 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.159679    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.247410 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.247383    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:11:06.247876 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.247855    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:11:06.248950 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.248935    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.249008 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:06.248965    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-134.us-west-1.compute.internal\": node \"ip-10-0-132-134.us-west-1.compute.internal\" not found"
Oct 30 21:11:06.259018 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.258991    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.260119 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.260104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.260171 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.260128    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d2a11b2240e0ab276a762b4e71b7c25e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"d2a11b2240e0ab276a762b4e71b7c25e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.260171 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.260143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.260251 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.260199    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.260251 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.260206    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.260251 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.260209    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d2a11b2240e0ab276a762b4e71b7c25e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"d2a11b2240e0ab276a762b4e71b7c25e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.319206 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.319175    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:11:06.319357 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.319284    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.374870 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.374784    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:11:06.412988 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.412957    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.416462 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.416428    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:11:06.804718 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.804638    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:11:06.805304 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.804764    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:11:06.805304 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.804761    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:11:06.805304 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.804761    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:11:06.838572 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.838542    2574 apiserver.go:52] "Watching apiserver"
Oct 30 21:11:06.857037 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:06.857009    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:11:07.081888 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.081784    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:06:07 +0000 UTC" deadline="2028-06-04 18:04:41.56797764 +0000 UTC"
Oct 30 21:11:07.081888 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.081831    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13988h53m34.486148777s"
Oct 30 21:11:07.109902 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.109879    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:11:07.110183 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.110164    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal","openshift-image-registry/node-ca-mwrs5","openshift-ovn-kubernetes/ovnkube-node-4lnfl","openshift-dns/node-resolver-nw865","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal","openshift-multus/multus-47z7f","openshift-multus/network-metrics-daemon-nbcn2","openshift-network-diagnostics/network-check-target-c65g6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p","openshift-multus/multus-additional-cni-plugins-pttss","kube-system/konnectivity-agent-zxr26","openshift-cluster-node-tuning-operator/tuned-kk2jz","openshift-network-operator/iptables-alerter-k4zbv","kube-system/global-pull-secret-syncer-r59mn"]
Oct 30 21:11:07.111738 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.111724    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:07.111817 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.111787    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:07.112534 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.112522    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.113591 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.113575    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.114697 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.114681    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.115555 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.115543    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.116548 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.116536    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.117558 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.117547    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.117612 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.117590    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:07.118604 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.118589    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.119885 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.119867    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.122822 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.122804    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:07.124091 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.124075    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.125210 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.125192    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:07.125290 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.125243    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:07.159804 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.159772    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:11:07.163911 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.163890    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8224a234-a815-4d78-9960-1c226c1224e4-host-slash\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.163983 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.163930    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-var-lib-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.163983 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.163956    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.163983 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.163978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-lib-modules\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.164098 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-var-lib-kubelet\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.164098 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164061    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e23751d7-2d21-4805-8a90-fd6276de3545-hosts-file\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.164192 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164098    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e23751d7-2d21-4805-8a90-fd6276de3545-tmp-dir\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.164192 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-kubelet\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.164282 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nssz5\" (UniqueName: \"kubernetes.io/projected/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-kube-api-access-nssz5\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:07.164282 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-script-lib\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.164282 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164260    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-etc-tuned\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.164401 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164305    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-conf-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.164401 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164337    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84ntf\" (UniqueName: \"kubernetes.io/projected/8224a234-a815-4d78-9960-1c226c1224e4-kube-api-access-84ntf\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.164401 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-dbus\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.164401 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164384    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-k8s-cni-cncf-io\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.164512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164416    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1a59846-fdda-4740-ada7-13de6a939e93-serviceca\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.164512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-kubelet\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.164512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164455    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.164512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-multus-daemon-config\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.164512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164484    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-multus-certs\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.164512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164497    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1a59846-fdda-4740-ada7-13de6a939e93-host\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.164512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164522    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-log-socket\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164535    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164553    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-node-log\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-registration-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164611    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-sys-fs\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164639    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-cnibin\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164685    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh9l9\" (UniqueName: \"kubernetes.io/projected/f804883e-c5eb-46e0-a6d2-64db96efa860-kube-api-access-zh9l9\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.164745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164734    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-socket-dir-parent\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164767    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06846bf4-72b9-4348-b4ae-5108221dbd0b-agent-certs\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164817    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8224a234-a815-4d78-9960-1c226c1224e4-iptables-alerter-script\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164841    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-netns\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164864    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-ovn\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164921    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164947    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwkmt\" (UniqueName: \"kubernetes.io/projected/99ddf84f-2164-4309-a9bb-30a816909083-kube-api-access-nwkmt\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-bin\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.164990    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-modprobe-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165015    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-cnibin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165044    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-cni-binary-copy\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165074    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-multus\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-etc-kubernetes\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165115    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px5f9\" (UniqueName: \"kubernetes.io/projected/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kube-api-access-px5f9\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165132    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-system-cni-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4b99\" (UniqueName: \"kubernetes.io/projected/e23751d7-2d21-4805-8a90-fd6276de3545-kube-api-access-z4b99\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.165204 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-slash\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-systemd\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165234    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-run\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165248    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-tmp\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165274    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-netd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165299    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-socket-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-device-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165344    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-conf\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-bin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165387    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-hostroot\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165412    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06846bf4-72b9-4348-b4ae-5108221dbd0b-konnectivity-ca\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165430    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-systemd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165444    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovn-node-metrics-cert\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165503    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysconfig\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165518    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-system-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.165691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165533    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt65s\" (UniqueName: \"kubernetes.io/projected/d1a59846-fdda-4740-ada7-13de6a939e93-kube-api-access-qt65s\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165569    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165587    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-sys\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165602    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-systemd-units\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165616    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-env-overrides\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165635    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-etc-selinux\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165653    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-kubelet-config\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165666    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-etc-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165679    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-config\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165705    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmf5z\" (UniqueName: \"kubernetes.io/projected/99dd82d0-8116-423a-8265-4c5b2f428ab8-kube-api-access-dmf5z\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165735    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-kubernetes\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxd9w\" (UniqueName: \"kubernetes.io/projected/8b042ce3-beb3-4075-8360-9849305e7e83-kube-api-access-wxd9w\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-netns\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165850    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.166197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165865    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-binary-copy\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.166612 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.165879    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-host\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.169326 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.169309    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cb6bb\""
Oct 30 21:11:07.170274 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.170260    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:11:07.170462 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.170446    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t4qh6\""
Oct 30 21:11:07.170540 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.170527    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-swkpp\""
Oct 30 21:11:07.171052 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.171039    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:11:07.171115 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.171103    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:11:07.171382 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.171371    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:11:07.172833 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.172820    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:11:07.263396 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.263371    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:11:07.263528 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.263412    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pg6dm"
Oct 30 21:11:07.266649 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.266605    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmf5z\" (UniqueName: \"kubernetes.io/projected/99dd82d0-8116-423a-8265-4c5b2f428ab8-kube-api-access-dmf5z\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.266771 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.266689    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.267036 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-kubernetes\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267098 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267041    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-kubernetes\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267098 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.267056    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:07.267098 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267063    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxd9w\" (UniqueName: \"kubernetes.io/projected/8b042ce3-beb3-4075-8360-9849305e7e83-kube-api-access-wxd9w\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267098 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-netns\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.267098 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267112    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.267143    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret podName:0492a73c-e41b-46c7-ad57-1f8220dfcf3a nodeName:}" failed. No retries permitted until 2026-10-30 21:11:07.767120265 +0000 UTC m=+2.227316904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret") pod "global-pull-secret-syncer-r59mn" (UID: "0492a73c-e41b-46c7-ad57-1f8220dfcf3a") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267158    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-binary-copy\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267170    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-netns\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-host\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.267250    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267254    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8224a234-a815-4d78-9960-1c226c1224e4-host-slash\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267289    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8224a234-a815-4d78-9960-1c226c1224e4-host-slash\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.267302    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs podName:fe2359dc-5156-4bd5-8e7f-1bce48d948e6 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:07.76728521 +0000 UTC m=+2.227481836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs") pod "network-metrics-daemon-nbcn2" (UID: "fe2359dc-5156-4bd5-8e7f-1bce48d948e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267293    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-host\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267340 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267340    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-var-lib-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267364    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267382    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-lib-modules\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267388    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-var-lib-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267404    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-var-lib-kubelet\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267425    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e23751d7-2d21-4805-8a90-fd6276de3545-hosts-file\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e23751d7-2d21-4805-8a90-fd6276de3545-tmp-dir\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267462    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-kubelet\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267478    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-var-lib-kubelet\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267480    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nssz5\" (UniqueName: \"kubernetes.io/projected/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-kube-api-access-nssz5\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267503    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-lib-modules\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267518    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267521    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-script-lib\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267546    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e23751d7-2d21-4805-8a90-fd6276de3545-hosts-file\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267578    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-etc-tuned\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267604    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-conf-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267627    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84ntf\" (UniqueName: \"kubernetes.io/projected/8224a234-a815-4d78-9960-1c226c1224e4-kube-api-access-84ntf\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267650    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-dbus\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.267872 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-k8s-cni-cncf-io\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267698    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1a59846-fdda-4740-ada7-13de6a939e93-serviceca\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267720    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-kubelet\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267522    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-kubelet\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267743    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267767    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-dbus\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-multus-daemon-config\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267671    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-conf-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267826    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-multus-certs\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267835    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e23751d7-2d21-4805-8a90-fd6276de3545-tmp-dir\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267831    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-k8s-cni-cncf-io\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267861    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-multus-certs\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267861    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-kubelet\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1a59846-fdda-4740-ada7-13de6a939e93-host\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267903    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267910    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-log-socket\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1a59846-fdda-4740-ada7-13de6a939e93-host\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.268745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267961    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-log-socket\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.267979    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268024    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268067    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-node-log\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268098    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-registration-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268105    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268114    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-sys-fs\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-cnibin\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268161    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-sys-fs\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268166    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh9l9\" (UniqueName: \"kubernetes.io/projected/f804883e-c5eb-46e0-a6d2-64db96efa860-kube-api-access-zh9l9\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268187    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-registration-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268204    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-socket-dir-parent\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268221    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-cnibin\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06846bf4-72b9-4348-b4ae-5108221dbd0b-agent-certs\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:07.270345 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268166    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-node-log\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268287    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8224a234-a815-4d78-9960-1c226c1224e4-iptables-alerter-script\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268292    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-socket-dir-parent\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268316    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268338    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268349    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-netns\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-netns\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-ovn\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268404    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268422    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268437    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwkmt\" (UniqueName: \"kubernetes.io/projected/99ddf84f-2164-4309-a9bb-30a816909083-kube-api-access-nwkmt\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268448    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268452    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-bin\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268471    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-ovn\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268487    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-modprobe-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268499    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268510    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-cnibin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-cni-binary-copy\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268536    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-bin\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268553    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:11:07.271144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268555    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-multus\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268577    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-etc-kubernetes\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268589    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268605    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px5f9\" (UniqueName: \"kubernetes.io/projected/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kube-api-access-px5f9\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268628    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-system-cni-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268640    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-cnibin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268650    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4b99\" (UniqueName: \"kubernetes.io/projected/e23751d7-2d21-4805-8a90-fd6276de3545-kube-api-access-z4b99\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268693    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268734    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268738    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-multus\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268749    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8224a234-a815-4d78-9960-1c226c1224e4-iptables-alerter-script\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268757    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-modprobe-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268828    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-slash\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268832    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-system-cni-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268846    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zwgx7\""
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268884    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-etc-kubernetes\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268897    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-systemd\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268922    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-run\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268899    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-slash\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.271890 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268944    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-tmp\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268970    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-netd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268984    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-run\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268991    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-socket-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269015    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-device-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269036    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-conf\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269051    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-bin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-hostroot\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269091    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269095    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269097    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-rvh9q\""
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269113    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06846bf4-72b9-4348-b4ae-5108221dbd0b-konnectivity-ca\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269090    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wtj2v\""
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269175    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-netd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269177    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-device-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269192    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-hostroot\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.268933    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-systemd\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269190    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-bin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269230    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-socket-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.272431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269253    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-systemd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovn-node-metrics-cert\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269279    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-conf\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269298    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269303    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-systemd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269327    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysconfig\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269348    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269371    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-system-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269398    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysconfig\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt65s\" (UniqueName: \"kubernetes.io/projected/d1a59846-fdda-4740-ada7-13de6a939e93-kube-api-access-qt65s\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269473    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-system-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269511    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-sys\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-systemd-units\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269607    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-env-overrides\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269618    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-sys\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269631    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-etc-selinux\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.272963 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269667    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-systemd-units\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269698    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-kubelet-config\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269724    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-etc-selinux\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-etc-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269760    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-etc-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269771    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-kubelet-config\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.269809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-config\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.270108    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-env-overrides\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.270216    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-config\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.270261    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.271440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-etc-tuned\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.271581    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-tmp\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.273428 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.271701    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovn-node-metrics-cert\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.278195 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.278174    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-binary-copy\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.278321 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.278195    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.278380 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.278322    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-multus-daemon-config\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.278380 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.278327    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1a59846-fdda-4740-ada7-13de6a939e93-serviceca\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.279677 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.279658    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-cni-binary-copy\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.281485 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.281468    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:11:07.281552 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.281468    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-x9kft\""
Oct 30 21:11:07.281552 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.281515    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xkwpv\""
Oct 30 21:11:07.282817 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.282789    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:11:07.290731 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.290715    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06846bf4-72b9-4348-b4ae-5108221dbd0b-konnectivity-ca\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:07.317233 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.317202    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:11:07.317233 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.317206    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:11:07.318516 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.318494    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:11:07.318625 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.318518    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:11:07.318625 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.318541    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:11:07.318705 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.318640    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87wxx\""
Oct 30 21:11:07.322933 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.322914    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:11:07.322933 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.322932    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:11:07.323193 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.322945    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gsn2n for pod openshift-network-diagnostics/network-check-target-c65g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:07.323193 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.323010    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n podName:4a5786b3-e466-4fc8-99c6-bc2876267401 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:07.822995969 +0000 UTC m=+2.283192604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gsn2n" (UniqueName: "kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n") pod "network-check-target-c65g6" (UID: "4a5786b3-e466-4fc8-99c6-bc2876267401") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:07.325440 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.325413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84ntf\" (UniqueName: \"kubernetes.io/projected/8224a234-a815-4d78-9960-1c226c1224e4-kube-api-access-84ntf\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.325687 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.325661    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nssz5\" (UniqueName: \"kubernetes.io/projected/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-kube-api-access-nssz5\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:07.325821 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.325769    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmf5z\" (UniqueName: \"kubernetes.io/projected/99dd82d0-8116-423a-8265-4c5b2f428ab8-kube-api-access-dmf5z\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.326480 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.326462    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt65s\" (UniqueName: \"kubernetes.io/projected/d1a59846-fdda-4740-ada7-13de6a939e93-kube-api-access-qt65s\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.327349 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.327331    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4b99\" (UniqueName: \"kubernetes.io/projected/e23751d7-2d21-4805-8a90-fd6276de3545-kube-api-access-z4b99\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.327440 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.327427    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh9l9\" (UniqueName: \"kubernetes.io/projected/f804883e-c5eb-46e0-a6d2-64db96efa860-kube-api-access-zh9l9\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.327633 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.327614    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px5f9\" (UniqueName: \"kubernetes.io/projected/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kube-api-access-px5f9\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.328472 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.328453    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-script-lib\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.328530 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.328508    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxd9w\" (UniqueName: \"kubernetes.io/projected/8b042ce3-beb3-4075-8360-9849305e7e83-kube-api-access-wxd9w\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.329060 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.329043    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwkmt\" (UniqueName: \"kubernetes.io/projected/99ddf84f-2164-4309-a9bb-30a816909083-kube-api-access-nwkmt\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.330430 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.330413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06846bf4-72b9-4348-b4ae-5108221dbd0b-agent-certs\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:07.444543 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.444477    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:11:07.468516 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.468468    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a11b2240e0ab276a762b4e71b7c25e.slice/crio-77a7dbfc9efa8f6193edf5e13551c0fb5315d9c0570130c9f44cf06d07323780 WatchSource:0}: Error finding container 77a7dbfc9efa8f6193edf5e13551c0fb5315d9c0570130c9f44cf06d07323780: Status 404 returned error can't find the container with id 77a7dbfc9efa8f6193edf5e13551c0fb5315d9c0570130c9f44cf06d07323780
Oct 30 21:11:07.469099 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.469073    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f2515e59c26a5d9f3c7f3a1c0a82c9.slice/crio-ced26a1d46e50a01617506a794137dc1eef61adb965bbc11eee56429b097313a WatchSource:0}: Error finding container ced26a1d46e50a01617506a794137dc1eef61adb965bbc11eee56429b097313a: Status 404 returned error can't find the container with id ced26a1d46e50a01617506a794137dc1eef61adb965bbc11eee56429b097313a
Oct 30 21:11:07.469646 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.469623    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8224a234_a815_4d78_9960_1c226c1224e4.slice/crio-d36cd8c38b4843bcb1a14c2991627a555637dac86ed828f75db32dc4cc8e93b7 WatchSource:0}: Error finding container d36cd8c38b4843bcb1a14c2991627a555637dac86ed828f75db32dc4cc8e93b7: Status 404 returned error can't find the container with id d36cd8c38b4843bcb1a14c2991627a555637dac86ed828f75db32dc4cc8e93b7
Oct 30 21:11:07.471507 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.471477    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:07.473559 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.473543    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:11:07.478516 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.478499    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nw865"
Oct 30 21:11:07.480869 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.480848    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99dd82d0_8116_423a_8265_4c5b2f428ab8.slice/crio-71604d9195264339523ac2dbfb9f5b3ea6ab2630c51ce057100d9e7e7e0e6a2e WatchSource:0}: Error finding container 71604d9195264339523ac2dbfb9f5b3ea6ab2630c51ce057100d9e7e7e0e6a2e: Status 404 returned error can't find the container with id 71604d9195264339523ac2dbfb9f5b3ea6ab2630c51ce057100d9e7e7e0e6a2e
Oct 30 21:11:07.486843 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.486817    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode23751d7_2d21_4805_8a90_fd6276de3545.slice/crio-e12a94578f03e5c94a708a1f65d538b8641c5615f95421e15c5406b1b7a952cf WatchSource:0}: Error finding container e12a94578f03e5c94a708a1f65d538b8641c5615f95421e15c5406b1b7a952cf: Status 404 returned error can't find the container with id e12a94578f03e5c94a708a1f65d538b8641c5615f95421e15c5406b1b7a952cf
Oct 30 21:11:07.500815 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.500783    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-47z7f"
Oct 30 21:11:07.505368 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.505346    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:11:07.509548 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.509530    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ddf84f_2164_4309_a9bb_30a816909083.slice/crio-fb134c75282a241532a64707fd797de266507d282984a9c0d671a6aa71cf1134 WatchSource:0}: Error finding container fb134c75282a241532a64707fd797de266507d282984a9c0d671a6aa71cf1134: Status 404 returned error can't find the container with id fb134c75282a241532a64707fd797de266507d282984a9c0d671a6aa71cf1134
Oct 30 21:11:07.514358 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.514337    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e4cd6ee_1f05_473a_b32c_d9a53a790541.slice/crio-c093e55397c67fa6a5936baf5661fb91db7544b1766d7b62798baac63bfecf32 WatchSource:0}: Error finding container c093e55397c67fa6a5936baf5661fb91db7544b1766d7b62798baac63bfecf32: Status 404 returned error can't find the container with id c093e55397c67fa6a5936baf5661fb91db7544b1766d7b62798baac63bfecf32
Oct 30 21:11:07.518690 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.518675    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:11:07.541661 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.541639    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:07.541753 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.541717    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:11:07.545179 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:07.545139    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:11:07.551281 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.551259    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06846bf4_72b9_4348_b4ae_5108221dbd0b.slice/crio-1900df2fe84054e14327a744aedb763f60f61f4af17f5c749f6396379cc8c797 WatchSource:0}: Error finding container 1900df2fe84054e14327a744aedb763f60f61f4af17f5c749f6396379cc8c797: Status 404 returned error can't find the container with id 1900df2fe84054e14327a744aedb763f60f61f4af17f5c749f6396379cc8c797
Oct 30 21:11:07.555339 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.555318    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf804883e_c5eb_46e0_a6d2_64db96efa860.slice/crio-101b3b78fdd561d363adeae439f500559787c01a606ed10d82262eb0e5bb4c88 WatchSource:0}: Error finding container 101b3b78fdd561d363adeae439f500559787c01a606ed10d82262eb0e5bb4c88: Status 404 returned error can't find the container with id 101b3b78fdd561d363adeae439f500559787c01a606ed10d82262eb0e5bb4c88
Oct 30 21:11:07.556052 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:07.556035    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b042ce3_beb3_4075_8360_9849305e7e83.slice/crio-bb2971c76256b2da72bb39df105ba2a7f34bbcd31dc47bee0b90ff780514b3d2 WatchSource:0}: Error finding container bb2971c76256b2da72bb39df105ba2a7f34bbcd31dc47bee0b90ff780514b3d2: Status 404 returned error can't find the container with id bb2971c76256b2da72bb39df105ba2a7f34bbcd31dc47bee0b90ff780514b3d2
Oct 30 21:11:07.558168 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.558147    2574 kuberuntime_manager.go:1707] "Unhandled Error" err="container tuned start failed in pod tuned-kk2jz_openshift-cluster-node-tuning-operator(8b042ce3-beb3-4075-8360-9849305e7e83): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Oct 30 21:11:07.558217 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:07.558180    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"tuned\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-cluster-node-tuning-operator/tuned-kk2jz" podUID="8b042ce3-beb3-4075-8360-9849305e7e83"
Oct 30 21:11:09.437595 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.437491    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:09.437595 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.437534    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:09.437768 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.437665    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:09.437768 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.437665    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:09.437768 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.437729    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret podName:0492a73c-e41b-46c7-ad57-1f8220dfcf3a nodeName:}" failed. No retries permitted until 2026-10-30 21:11:10.437714153 +0000 UTC m=+3.233498821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret") pod "global-pull-secret-syncer-r59mn" (UID: "0492a73c-e41b-46c7-ad57-1f8220dfcf3a") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:09.437768 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.437744    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs podName:fe2359dc-5156-4bd5-8e7f-1bce48d948e6 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:10.437737656 +0000 UTC m=+3.233522319 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs") pod "network-metrics-daemon-nbcn2" (UID: "fe2359dc-5156-4bd5-8e7f-1bce48d948e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:09.538722 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.538677    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:09.539351 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.538842    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:11:09.539351 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.538861    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:11:09.539351 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.538871    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gsn2n for pod openshift-network-diagnostics/network-check-target-c65g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:09.539351 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.538919    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n podName:4a5786b3-e466-4fc8-99c6-bc2876267401 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:10.538906233 +0000 UTC m=+3.334690896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsn2n" (UniqueName: "kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n") pod "network-check-target-c65g6" (UID: "4a5786b3-e466-4fc8-99c6-bc2876267401") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:09.650584 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.650533    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kk2jz" event={"ID":"8b042ce3-beb3-4075-8360-9849305e7e83","Type":"ContainerStarted","Data":"bb2971c76256b2da72bb39df105ba2a7f34bbcd31dc47bee0b90ff780514b3d2"}
Oct 30 21:11:09.651633 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.651601    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zxr26" event={"ID":"06846bf4-72b9-4348-b4ae-5108221dbd0b","Type":"ContainerStarted","Data":"1900df2fe84054e14327a744aedb763f60f61f4af17f5c749f6396379cc8c797"}
Oct 30 21:11:09.652172 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:09.652147    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"tuned\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-cluster-node-tuning-operator/tuned-kk2jz" podUID="8b042ce3-beb3-4075-8360-9849305e7e83"
Oct 30 21:11:09.652657 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.652641    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mwrs5" event={"ID":"d1a59846-fdda-4740-ada7-13de6a939e93","Type":"ContainerStarted","Data":"880a3aa2d1320f60b4091c342d93ce7938629e06079302dd3592d8a58d7029eb"}
Oct 30 21:11:09.653664 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.653637    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p" event={"ID":"1e4cd6ee-1f05-473a-b32c-d9a53a790541","Type":"ContainerStarted","Data":"c093e55397c67fa6a5936baf5661fb91db7544b1766d7b62798baac63bfecf32"}
Oct 30 21:11:09.654589 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.654572    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-47z7f" event={"ID":"99ddf84f-2164-4309-a9bb-30a816909083","Type":"ContainerStarted","Data":"fb134c75282a241532a64707fd797de266507d282984a9c0d671a6aa71cf1134"}
Oct 30 21:11:09.655490 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.655467    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"d2a11b2240e0ab276a762b4e71b7c25e","Type":"ContainerStarted","Data":"77a7dbfc9efa8f6193edf5e13551c0fb5315d9c0570130c9f44cf06d07323780"}
Oct 30 21:11:09.656432 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.656414    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pttss" event={"ID":"f804883e-c5eb-46e0-a6d2-64db96efa860","Type":"ContainerStarted","Data":"101b3b78fdd561d363adeae439f500559787c01a606ed10d82262eb0e5bb4c88"}
Oct 30 21:11:09.657370 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.657353    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nw865" event={"ID":"e23751d7-2d21-4805-8a90-fd6276de3545","Type":"ContainerStarted","Data":"e12a94578f03e5c94a708a1f65d538b8641c5615f95421e15c5406b1b7a952cf"}
Oct 30 21:11:09.658405 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.658388    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"71604d9195264339523ac2dbfb9f5b3ea6ab2630c51ce057100d9e7e7e0e6a2e"}
Oct 30 21:11:09.659427 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.659402    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4zbv" event={"ID":"8224a234-a815-4d78-9960-1c226c1224e4","Type":"ContainerStarted","Data":"d36cd8c38b4843bcb1a14c2991627a555637dac86ed828f75db32dc4cc8e93b7"}
Oct 30 21:11:09.660745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.660719    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"f9f2515e59c26a5d9f3c7f3a1c0a82c9","Type":"ContainerStarted","Data":"ced26a1d46e50a01617506a794137dc1eef61adb965bbc11eee56429b097313a"}
Oct 30 21:11:09.686861 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.686845    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:11:09.716781 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.716728    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:11:09.738456 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.738435    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:11:09.929957 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.929896    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:06:08 +0000 UTC" deadline="2028-04-14 14:25:20.407140564 +0000 UTC"
Oct 30 21:11:09.929957 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:09.929952    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12761h14m10.477191114s"
Oct 30 21:11:10.446757 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:10.446716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:10.446947 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:10.446772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:10.446947 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.446933    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:10.447044 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.446991    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs podName:fe2359dc-5156-4bd5-8e7f-1bce48d948e6 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:12.446974858 +0000 UTC m=+5.242759524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs") pod "network-metrics-daemon-nbcn2" (UID: "fe2359dc-5156-4bd5-8e7f-1bce48d948e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:10.447114 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.447050    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:10.447114 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.447083    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret podName:0492a73c-e41b-46c7-ad57-1f8220dfcf3a nodeName:}" failed. No retries permitted until 2026-10-30 21:11:12.44707223 +0000 UTC m=+5.242856907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret") pod "global-pull-secret-syncer-r59mn" (UID: "0492a73c-e41b-46c7-ad57-1f8220dfcf3a") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:10.547769 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:10.547414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:10.547769 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.547594    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:11:10.547769 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.547613    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:11:10.547769 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.547625    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gsn2n for pod openshift-network-diagnostics/network-check-target-c65g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:10.547769 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.547683    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n podName:4a5786b3-e466-4fc8-99c6-bc2876267401 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:12.547664172 +0000 UTC m=+5.343448849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsn2n" (UniqueName: "kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n") pod "network-check-target-c65g6" (UID: "4a5786b3-e466-4fc8-99c6-bc2876267401") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:10.646690 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:10.646114    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:10.646690 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.646249    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:10.646690 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:10.646335    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:10.646690 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.646407    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:10.646690 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:10.646466    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:10.646690 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:10.646607    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:10.930951 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:10.930864    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:06:08 +0000 UTC" deadline="2028-07-23 15:29:09.002841456 +0000 UTC"
Oct 30 21:11:10.930951 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:10.930906    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15162h17m58.071940187s"
Oct 30 21:11:12.465042 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:12.465004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:12.465669 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:12.465107    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:12.465669 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.465240    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:12.465669 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.465298    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret podName:0492a73c-e41b-46c7-ad57-1f8220dfcf3a nodeName:}" failed. No retries permitted until 2026-10-30 21:11:16.46528186 +0000 UTC m=+9.261066527 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret") pod "global-pull-secret-syncer-r59mn" (UID: "0492a73c-e41b-46c7-ad57-1f8220dfcf3a") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:12.465907 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.465703    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:12.465907 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.465777    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs podName:fe2359dc-5156-4bd5-8e7f-1bce48d948e6 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:16.465738345 +0000 UTC m=+9.261523015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs") pod "network-metrics-daemon-nbcn2" (UID: "fe2359dc-5156-4bd5-8e7f-1bce48d948e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:12.565940 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:12.565895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:12.566075 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.566054    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:11:12.566075 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.566070    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:11:12.566177 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.566079    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gsn2n for pod openshift-network-diagnostics/network-check-target-c65g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:12.566177 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.566134    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n podName:4a5786b3-e466-4fc8-99c6-bc2876267401 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:16.566117369 +0000 UTC m=+9.361902045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsn2n" (UniqueName: "kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n") pod "network-check-target-c65g6" (UID: "4a5786b3-e466-4fc8-99c6-bc2876267401") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:12.646848 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:12.646814    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:12.647001 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.646953    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:12.647077 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:12.647011    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:12.647135 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.647077    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:12.647135 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:12.647125    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:12.647266 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:12.647226    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:13.689248 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:13.688048    2574 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="799dd8e44f7cff32935c1786adedf1d6ecd0f7f7e6313a66943acaee1e9712e8" exitCode=0
Oct 30 21:11:13.689248 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:13.688393    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pttss" event={"ID":"f804883e-c5eb-46e0-a6d2-64db96efa860","Type":"ContainerDied","Data":"799dd8e44f7cff32935c1786adedf1d6ecd0f7f7e6313a66943acaee1e9712e8"}
Oct 30 21:11:13.706742 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:13.706708    2574 generic.go:356] "Generic (PLEG): container finished" podID="f9f2515e59c26a5d9f3c7f3a1c0a82c9" containerID="99737064de0e9db30413d2a7ec605a1e581f4b4428580e4e187bfe5163fbffa6" exitCode=0
Oct 30 21:11:13.706874 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:13.706760    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"f9f2515e59c26a5d9f3c7f3a1c0a82c9","Type":"ContainerDied","Data":"99737064de0e9db30413d2a7ec605a1e581f4b4428580e4e187bfe5163fbffa6"}
Oct 30 21:11:14.646510 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:14.646480    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:14.646632 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:14.646610    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:14.646719 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:14.646704    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:14.646809 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:14.646787    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:14.646868 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:14.646859    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:14.646971 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:14.646951    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:14.713745 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:14.713002    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"d2a11b2240e0ab276a762b4e71b7c25e","Type":"ContainerStarted","Data":"d64bb761b6211c452c070f28bf0af5cfc74c99b19a6b0d2ed6011a759157f647"}
Oct 30 21:11:14.717145 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:14.716699    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"f9f2515e59c26a5d9f3c7f3a1c0a82c9","Type":"ContainerStarted","Data":"240457ed84b22bb167b212ea428802c7ae68e1d45b5cff3fed79e7b5722eb43d"}
Oct 30 21:11:14.719412 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:14.718921    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zxr26" event={"ID":"06846bf4-72b9-4348-b4ae-5108221dbd0b","Type":"ContainerStarted","Data":"c1376aa4f883352d0315f1b3b0c1212ac15f792fd541062dc9554a9d0789dc13"}
Oct 30 21:11:14.946463 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:14.945886    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zxr26" podStartSLOduration=3.543925125 podStartE2EDuration="7.94587016s" totalImagesPullingTime="4.401945035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:14.943854691 +0000 UTC m=+7.739639376" watchObservedRunningTime="2026-10-30 21:11:14.94587016 +0000 UTC m=+7.741654885"
Oct 30 21:11:14.946463 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:14.946063    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal" podStartSLOduration=6.946057847 podStartE2EDuration="6.946057847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:14.826891517 +0000 UTC m=+7.622676206" watchObservedRunningTime="2026-10-30 21:11:14.946057847 +0000 UTC m=+7.741842567"
Oct 30 21:11:15.722902 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:15.722823    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mwrs5" event={"ID":"d1a59846-fdda-4740-ada7-13de6a939e93","Type":"ContainerStarted","Data":"588a830e49f8ace19ac27c0bbdb5189f76275ca630741ed93ead51315a25f165"}
Oct 30 21:11:15.725384 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:15.725352    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p" event={"ID":"1e4cd6ee-1f05-473a-b32c-d9a53a790541","Type":"ContainerStarted","Data":"8caed8e7b37e16f22083ef824908a463a1a63a0414122879a52e621fad7a94c8"}
Oct 30 21:11:15.839070 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:15.839000    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal" podStartSLOduration=8.838981839 podStartE2EDuration="8.838981839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:15.061879886 +0000 UTC m=+7.857664571" watchObservedRunningTime="2026-10-30 21:11:15.838981839 +0000 UTC m=+8.634766523"
Oct 30 21:11:16.499412 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:16.499370    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:16.499572 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:16.499432    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:16.499642 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.499601    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:16.499680 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.499668    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs podName:fe2359dc-5156-4bd5-8e7f-1bce48d948e6 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:24.499650274 +0000 UTC m=+17.295434960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs") pod "network-metrics-daemon-nbcn2" (UID: "fe2359dc-5156-4bd5-8e7f-1bce48d948e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:16.500135 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.500110    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:16.500252 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.500170    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret podName:0492a73c-e41b-46c7-ad57-1f8220dfcf3a nodeName:}" failed. No retries permitted until 2026-10-30 21:11:24.500155153 +0000 UTC m=+17.295939829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret") pod "global-pull-secret-syncer-r59mn" (UID: "0492a73c-e41b-46c7-ad57-1f8220dfcf3a") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:16.600710 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:16.600673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:16.600896 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.600878    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:11:16.600952 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.600901    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:11:16.600952 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.600912    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gsn2n for pod openshift-network-diagnostics/network-check-target-c65g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:16.601033 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.600968    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n podName:4a5786b3-e466-4fc8-99c6-bc2876267401 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:24.6009553 +0000 UTC m=+17.396739962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsn2n" (UniqueName: "kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n") pod "network-check-target-c65g6" (UID: "4a5786b3-e466-4fc8-99c6-bc2876267401") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:16.646074 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:16.645996    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:16.646276 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.646101    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:16.646276 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:16.646154    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:16.646276 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.646227    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:16.646445 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:16.646293    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:16.646445 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:16.646353    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:17.007602 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.007575    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:11:17.591639 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.591555    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:11:17.007598109Z","UUID":"714f09e2-3759-471f-b539-b7b848187d88","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:11:17.592741 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.592721    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:11:17.592741 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.592747    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:11:17.730814 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.730771    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p" event={"ID":"1e4cd6ee-1f05-473a-b32c-d9a53a790541","Type":"ContainerStarted","Data":"48eb4776ebd3177140ef46ad42ffd74f8381bb8947787205bc85e0014ae9ec48"}
Oct 30 21:11:17.732091 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.732065    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-47z7f" event={"ID":"99ddf84f-2164-4309-a9bb-30a816909083","Type":"ContainerStarted","Data":"6eea3bc0bb61726efad089fb972bbc5329e86a9152e31428d3452d96cc98a65f"}
Oct 30 21:11:17.733770 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.733740    2574 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="f94ee1485369c6f86ba6c4c533ea02e208bc07e5f7606baf5b9ffe52c62bb2d8" exitCode=0
Oct 30 21:11:17.733867 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.733782    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pttss" event={"ID":"f804883e-c5eb-46e0-a6d2-64db96efa860","Type":"ContainerDied","Data":"f94ee1485369c6f86ba6c4c533ea02e208bc07e5f7606baf5b9ffe52c62bb2d8"}
Oct 30 21:11:17.735161 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.735140    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nw865" event={"ID":"e23751d7-2d21-4805-8a90-fd6276de3545","Type":"ContainerStarted","Data":"5b70dfdefbafb0710da0b0cd09974f6eaadbeef767feac0d2c2f37d341c38b57"}
Oct 30 21:11:17.848912 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.848813    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mwrs5" podStartSLOduration=5.411834599 podStartE2EDuration="10.848800759s" totalImagesPullingTime="5.43696616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:15.840230227 +0000 UTC m=+8.636014911" watchObservedRunningTime="2026-10-30 21:11:17.848800759 +0000 UTC m=+10.644585442"
Oct 30 21:11:17.849077 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.849057    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-47z7f" podStartSLOduration=3.298892045 podStartE2EDuration="10.849053888s" totalImagesPullingTime="7.550161843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:17.848688314 +0000 UTC m=+10.644472997" watchObservedRunningTime="2026-10-30 21:11:17.849053888 +0000 UTC m=+10.644838572"
Oct 30 21:11:17.962747 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:17.962698    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-nw865" podStartSLOduration=3.26327398 podStartE2EDuration="10.962683084s" totalImagesPullingTime="7.699409104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:17.962456071 +0000 UTC m=+10.758240757" watchObservedRunningTime="2026-10-30 21:11:17.962683084 +0000 UTC m=+10.758467766"
Oct 30 21:11:18.646270 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.646237    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:18.646765 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.646281    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:18.646765 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:18.646347    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:18.646765 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:18.646376    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:18.646765 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.646406    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:18.646765 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:18.646476    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:18.731010 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.730930    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:18.731622 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.731602    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:18.740286 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.740242    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p" event={"ID":"1e4cd6ee-1f05-473a-b32c-d9a53a790541","Type":"ContainerStarted","Data":"cea5cd1dbea1a7217bbfe401a18656dc42dbc45d7bb8bb67a63009a26fe3f344"}
Oct 30 21:11:18.741862 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.741832    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4zbv" event={"ID":"8224a234-a815-4d78-9960-1c226c1224e4","Type":"ContainerStarted","Data":"fc6a26b1c0a38fe066c5cca3536b1cb74f322f1448809caedf89b1403a5ccf41"}
Oct 30 21:11:18.742376 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.742354    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:18.742744 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.742729    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:11:18.958986 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:18.958939    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k4zbv" podStartSLOduration=4.591362428 podStartE2EDuration="11.958923703s" totalImagesPullingTime="7.367561275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:18.958367043 +0000 UTC m=+11.754151727" watchObservedRunningTime="2026-10-30 21:11:18.958923703 +0000 UTC m=+11.754708386"
Oct 30 21:11:19.076521 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:19.074836    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p" podStartSLOduration=3.138047266 podStartE2EDuration="12.074817832s" totalImagesPullingTime="8.936770566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:19.074136955 +0000 UTC m=+11.869921641" watchObservedRunningTime="2026-10-30 21:11:19.074817832 +0000 UTC m=+11.870602517"
Oct 30 21:11:19.746643 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:19.746600    2574 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="40d9d39791eab5574daf77751e02c5530f0ac0a5e47477cb90c9c85c6aeaa459" exitCode=0
Oct 30 21:11:19.747751 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:19.747722    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pttss" event={"ID":"f804883e-c5eb-46e0-a6d2-64db96efa860","Type":"ContainerDied","Data":"40d9d39791eab5574daf77751e02c5530f0ac0a5e47477cb90c9c85c6aeaa459"}
Oct 30 21:11:20.646384 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:20.646342    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:20.646613 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:20.646411    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:20.646613 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:20.646542    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:20.646613 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:20.646551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:20.646781 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:20.646653    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:20.646781 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:20.646739    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:21.753522 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:21.753480    2574 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="316cccf27fb06dacfd9f62ed86981bc0c9502fc72e7e2de9c5c138454eb5ebde" exitCode=0
Oct 30 21:11:21.754329 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:21.753537    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pttss" event={"ID":"f804883e-c5eb-46e0-a6d2-64db96efa860","Type":"ContainerDied","Data":"316cccf27fb06dacfd9f62ed86981bc0c9502fc72e7e2de9c5c138454eb5ebde"}
Oct 30 21:11:22.646277 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:22.646236    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:22.646472 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:22.646291    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:22.646472 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:22.646309    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:22.646782 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:22.646754    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:22.646967 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:22.646937    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:22.647051 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:22.647031    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:24.560841 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:24.560757    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:24.560841 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:24.560802    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:24.561308 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.560919    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:24.561308 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.560930    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:24.561308 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.560984    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs podName:fe2359dc-5156-4bd5-8e7f-1bce48d948e6 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:40.560963757 +0000 UTC m=+33.356748436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs") pod "network-metrics-daemon-nbcn2" (UID: "fe2359dc-5156-4bd5-8e7f-1bce48d948e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:24.561308 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.561004    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret podName:0492a73c-e41b-46c7-ad57-1f8220dfcf3a nodeName:}" failed. No retries permitted until 2026-10-30 21:11:40.560994149 +0000 UTC m=+33.356778822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret") pod "global-pull-secret-syncer-r59mn" (UID: "0492a73c-e41b-46c7-ad57-1f8220dfcf3a") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:24.646902 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:24.646869    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:24.647085 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:24.646907    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:24.647085 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:24.646907    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:24.647085 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.647003    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:24.647285 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.647100    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:24.647285 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.647198    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:24.662137 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:24.662097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:24.662324 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.662309    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:11:24.662383 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.662332    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:11:24.662383 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.662344    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gsn2n for pod openshift-network-diagnostics/network-check-target-c65g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:24.662485 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:24.662394    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n podName:4a5786b3-e466-4fc8-99c6-bc2876267401 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:40.662377201 +0000 UTC m=+33.458161866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsn2n" (UniqueName: "kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n") pod "network-check-target-c65g6" (UID: "4a5786b3-e466-4fc8-99c6-bc2876267401") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:24.763343 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:24.763305    2574 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="2d09aa7691466b9fc3475ff1c96a862f0bd19346c38165492a22cf6e8d491231" exitCode=0
Oct 30 21:11:24.763509 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:24.763359    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pttss" event={"ID":"f804883e-c5eb-46e0-a6d2-64db96efa860","Type":"ContainerDied","Data":"2d09aa7691466b9fc3475ff1c96a862f0bd19346c38165492a22cf6e8d491231"}
Oct 30 21:11:25.769741 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:25.769451    2574 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="5542bd59062a1711cc5fab143cec96e13e8a35145fe5fb9018b50387854b5bdf" exitCode=0
Oct 30 21:11:25.769741 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:25.769509    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pttss" event={"ID":"f804883e-c5eb-46e0-a6d2-64db96efa860","Type":"ContainerDied","Data":"5542bd59062a1711cc5fab143cec96e13e8a35145fe5fb9018b50387854b5bdf"}
Oct 30 21:11:26.646952 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:26.646904    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:26.646952 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:26.646957    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:26.647184 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:26.646980    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:26.647184 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:26.647087    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:26.647184 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:26.647159    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:26.647356 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:26.647245    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:26.776759 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:26.776714    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pttss" event={"ID":"f804883e-c5eb-46e0-a6d2-64db96efa860","Type":"ContainerStarted","Data":"33defaa778152f51a6ab2ed08764eff6c125ab680c36100332f678ec26834860"}
Oct 30 21:11:26.926919 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:26.926874    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pttss" podStartSLOduration=7.962954185 podStartE2EDuration="19.92686045s" totalImagesPullingTime="11.963906265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:26.926473281 +0000 UTC m=+19.722257960" watchObservedRunningTime="2026-10-30 21:11:26.92686045 +0000 UTC m=+19.722645135"
Oct 30 21:11:27.782298 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:27.782073    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"f1123db978accb71f4c798b4f5758f577f68a425f24bd5a3aec2a6dac0330fa7"}
Oct 30 21:11:27.782777 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:27.782361    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"71c665abc9b999688f062511d00e7c04c8254bc6a5ed775541fdc2ebe4e5756e"}
Oct 30 21:11:27.782777 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:27.782428    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"40a29dad81ed9574893540f56ded331095c361da9aa7c625342020960a8a47df"}
Oct 30 21:11:27.782777 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:27.782486    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"74cb329e244270912335dde87b11f7bde8218ecd5bb117f26b1b0de9c634e8ae"}
Oct 30 21:11:27.782777 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:27.782529    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"c565ca5e7c541230218f142c093941583e056df94c9cddabb0175d1fd7f869f4"}
Oct 30 21:11:27.782777 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:27.782577    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"a23d917b1ce343dbf64da4b0efbadd107c87c537cc243538075fa082e3fdd1be"}
Oct 30 21:11:28.646522 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:28.646495    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:28.646522 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:28.646518    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:28.646715 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:28.646519    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:28.646715 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:28.646589    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:28.646715 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:28.646678    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:28.646802 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:28.646742    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:28.785854 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:28.785821    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kk2jz" event={"ID":"8b042ce3-beb3-4075-8360-9849305e7e83","Type":"ContainerStarted","Data":"f2f4b19f42de7800bcd141936da58aeb22a22b75afced351101be4f2fe456c73"}
Oct 30 21:11:28.899643 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:28.899555    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kk2jz" podStartSLOduration=3.146790233 podStartE2EDuration="21.899541916s" totalImagesPullingTime="18.752751683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-10-30 21:11:28.899336573 +0000 UTC m=+21.695121257" watchObservedRunningTime="2026-10-30 21:11:28.899541916 +0000 UTC m=+21.695326601"
Oct 30 21:11:29.791197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:29.791165    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"e4beff1afcfdfd1c4ec2127fa8e2724640116983824d7e95a3efa5ce15b17805"}
Oct 30 21:11:30.157643 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:30.157572    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nw865_e23751d7-2d21-4805-8a90-fd6276de3545/dns-node-resolver/0.log"
Oct 30 21:11:30.646194 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:30.646155    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:30.646194 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:30.646155    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:30.646479 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:30.646261    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:30.646479 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:30.646272    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:30.646479 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:30.646343    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:30.646479 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:30.646405    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:30.995956 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:30.995929    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mwrs5_d1a59846-fdda-4740-ada7-13de6a939e93/node-ca/0.log"
Oct 30 21:11:31.799194 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:31.799159    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" event={"ID":"99dd82d0-8116-423a-8265-4c5b2f428ab8","Type":"ContainerStarted","Data":"e2d3269dfc0ddb750da6391ead6c5c2838ce880fe9fd736cd46f3deefc260de6"}
Oct 30 21:11:31.799415 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:31.799396    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:31.815049 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:31.815023    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:32.071137 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:32.071048    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl" podStartSLOduration=7.353835707 podStartE2EDuration="25.07103507s" totalImagesPullingTime="17.717199363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:32.070870401 +0000 UTC m=+24.866655087" watchObservedRunningTime="2026-10-30 21:11:32.07103507 +0000 UTC m=+24.866819755"
Oct 30 21:11:32.646765 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:32.646732    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:32.646956 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:32.646804    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:32.646956 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:32.646832    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:32.646956 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:32.646917    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:32.647120 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:32.646983    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:32.647120 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:32.647057    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:32.800950 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:32.800924    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:11:32.801353 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:32.801336    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:32.816913 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:32.816893    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:33.802682 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:33.802657    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:11:34.235338 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:34.235300    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:11:34.646964 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:34.646879    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:34.647144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:34.646989    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:34.647144 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:34.647002    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:34.647144 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:34.647092    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:34.647144 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:34.647138    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:34.647374 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:34.647222    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:36.646430 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:36.646346    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:36.646430 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:36.646381    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:36.646834 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:36.646367    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:36.646834 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:36.646476    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:36.646834 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:36.646537    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:36.646834 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:36.646579    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:38.647091 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:38.646909    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:38.647556 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:38.646956    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:38.647556 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:38.647159    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:38.647556 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:38.646961    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:38.647556 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:38.647244    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:38.647556 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:38.647344    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:40.579543 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:40.579502    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:40.579543 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:40.579547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:40.579977 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.579650    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:40.579977 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.579714    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret podName:0492a73c-e41b-46c7-ad57-1f8220dfcf3a nodeName:}" failed. No retries permitted until 2026-10-30 21:12:12.579700296 +0000 UTC m=+65.375484973 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret") pod "global-pull-secret-syncer-r59mn" (UID: "0492a73c-e41b-46c7-ad57-1f8220dfcf3a") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:11:40.579977 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.579653    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:40.579977 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.579788    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs podName:fe2359dc-5156-4bd5-8e7f-1bce48d948e6 nodeName:}" failed. No retries permitted until 2026-10-30 21:12:12.579775104 +0000 UTC m=+65.375559767 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs") pod "network-metrics-daemon-nbcn2" (UID: "fe2359dc-5156-4bd5-8e7f-1bce48d948e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:11:40.646389 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:40.646355    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:40.646389 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:40.646380    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:40.646569 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.646463    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:40.646569 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:40.646500    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:40.646569 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.646560    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:40.646657 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.646600    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:40.680535 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:40.680505    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:40.680649 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.680619    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:11:40.680649 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.680631    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:11:40.680649 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.680639    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gsn2n for pod openshift-network-diagnostics/network-check-target-c65g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:40.680741 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:40.680679    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n podName:4a5786b3-e466-4fc8-99c6-bc2876267401 nodeName:}" failed. No retries permitted until 2026-10-30 21:12:12.680668312 +0000 UTC m=+65.476452974 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsn2n" (UniqueName: "kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n") pod "network-check-target-c65g6" (UID: "4a5786b3-e466-4fc8-99c6-bc2876267401") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:11:42.651528 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:42.651492    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:42.656349 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:42.652106    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:42.656531 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:42.652171    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:42.656656 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:42.656627    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:42.656719 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:42.656626    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:42.656776 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:42.656725    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:44.224275 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:44.224197    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r59mn"]
Oct 30 21:11:44.224856 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:44.224383    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:44.225060 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:44.225023    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:44.225060 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:44.225056    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c65g6"]
Oct 30 21:11:44.225282 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:44.225150    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:44.225282 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:44.225259    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:44.225889 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:44.225870    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nbcn2"]
Oct 30 21:11:44.225976 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:44.225964    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:44.226094 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:44.226075    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:45.646132 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:45.646094    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:45.646618 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:45.646110    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:45.646618 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:45.646254    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:45.646618 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:45.646306    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:46.646561 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:46.646526    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:46.646979 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:46.646633    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r59mn" podUID="0492a73c-e41b-46c7-ad57-1f8220dfcf3a"
Oct 30 21:11:47.649811 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:47.649782    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:47.650198 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:47.649823    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:47.650198 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:47.649931    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbcn2" podUID="fe2359dc-5156-4bd5-8e7f-1bce48d948e6"
Oct 30 21:11:47.650198 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:11:47.650031    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c65g6" podUID="4a5786b3-e466-4fc8-99c6-bc2876267401"
Oct 30 21:11:48.519774 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.519693    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:11:48.519912 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.519846    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:11:48.614989 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.614955    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sjfxt"]
Oct 30 21:11:48.694068 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.694035    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fvn9q"]
Oct 30 21:11:48.694540 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.694149    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.710066 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.710047    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:11:48.710177 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.710073    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zwzcq"]
Oct 30 21:11:48.710252 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.710187    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:11:48.723962 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.723941    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sjfxt"]
Oct 30 21:11:48.724097 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.723984    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fvn9q"]
Oct 30 21:11:48.724097 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.724064    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.744486 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.744462    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zwzcq"]
Oct 30 21:11:48.746264 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.746235    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:11:48.746409 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.746396    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dssbm\""
Oct 30 21:11:48.746476 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.746425    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:11:48.840512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.840433    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:11:48.840512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.840440    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:11:48.840512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.840466    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:11:48.840512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.840476    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:11:48.840512 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.840481    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:11:48.842802 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.842774    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b61fec4-1b8d-449c-b251-b48c3306d351-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.842880 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.842825    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b7c30e-9edc-43f6-ba65-279d40a3267e-metrics-tls\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.842915 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.842888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.843005 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.842992    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b7c30e-9edc-43f6-ba65-279d40a3267e-config-volume\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.843046 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.843011    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8b7c30e-9edc-43f6-ba65-279d40a3267e-tmp-dir\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.843046 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.843025    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w9zx\" (UniqueName: \"kubernetes.io/projected/b8b7c30e-9edc-43f6-ba65-279d40a3267e-kube-api-access-9w9zx\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.843046 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.843042    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2238e566-a39a-47ff-8f14-a27a404bc255-cert\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:11:48.843155 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.843060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b61fec4-1b8d-449c-b251-b48c3306d351-data-volume\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.843155 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.843102    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mbqj\" (UniqueName: \"kubernetes.io/projected/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-api-access-9mbqj\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.843226 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.843184    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9pvw\" (UniqueName: \"kubernetes.io/projected/2238e566-a39a-47ff-8f14-a27a404bc255-kube-api-access-c9pvw\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:11:48.843264 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.843227    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b61fec4-1b8d-449c-b251-b48c3306d351-crio-socket\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.846593 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.846577    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:11:48.846746 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.846718    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-k8b2g\""
Oct 30 21:11:48.846746 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.846718    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:11:48.846914 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.846763    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tklwz\""
Oct 30 21:11:48.846914 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.846779    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:11:48.846914 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.846836    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c2dlm\""
Oct 30 21:11:48.944368 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b7c30e-9edc-43f6-ba65-279d40a3267e-config-volume\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.944368 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944375    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8b7c30e-9edc-43f6-ba65-279d40a3267e-tmp-dir\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.944624 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w9zx\" (UniqueName: \"kubernetes.io/projected/b8b7c30e-9edc-43f6-ba65-279d40a3267e-kube-api-access-9w9zx\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.944624 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944424    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2238e566-a39a-47ff-8f14-a27a404bc255-cert\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:11:48.944624 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944448    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b61fec4-1b8d-449c-b251-b48c3306d351-data-volume\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.944624 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mbqj\" (UniqueName: \"kubernetes.io/projected/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-api-access-9mbqj\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.944624 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9pvw\" (UniqueName: \"kubernetes.io/projected/2238e566-a39a-47ff-8f14-a27a404bc255-kube-api-access-c9pvw\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:11:48.944624 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b61fec4-1b8d-449c-b251-b48c3306d351-crio-socket\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.944624 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b61fec4-1b8d-449c-b251-b48c3306d351-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.944624 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944581    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b7c30e-9edc-43f6-ba65-279d40a3267e-metrics-tls\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.944999 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944628    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.944999 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944845    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b61fec4-1b8d-449c-b251-b48c3306d351-data-volume\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.944999 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.944885    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8b7c30e-9edc-43f6-ba65-279d40a3267e-tmp-dir\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.945155 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.945043    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b61fec4-1b8d-449c-b251-b48c3306d351-crio-socket\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.945155 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.945109    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b7c30e-9edc-43f6-ba65-279d40a3267e-config-volume\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:48.945296 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.945248    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.948692 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.948587    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b61fec4-1b8d-449c-b251-b48c3306d351-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:48.948785 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.948699    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2238e566-a39a-47ff-8f14-a27a404bc255-cert\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:11:48.948785 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:48.948591    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b7c30e-9edc-43f6-ba65-279d40a3267e-metrics-tls\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:49.001535 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.001492    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mbqj\" (UniqueName: \"kubernetes.io/projected/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-api-access-9mbqj\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:49.001678 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.001567    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w9zx\" (UniqueName: \"kubernetes.io/projected/b8b7c30e-9edc-43f6-ba65-279d40a3267e-kube-api-access-9w9zx\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:49.001678 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.001605    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9pvw\" (UniqueName: \"kubernetes.io/projected/2238e566-a39a-47ff-8f14-a27a404bc255-kube-api-access-c9pvw\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:11:49.030657 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.030627    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:49.052247 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.052185    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:11:49.127822 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.127768    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:11:49.497216 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.497180    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fvn9q"]
Oct 30 21:11:49.500510 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.500487    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sjfxt"]
Oct 30 21:11:49.502002 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:49.501972    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2238e566_a39a_47ff_8f14_a27a404bc255.slice/crio-b6831c3f2c9e20d74d9c8847890d323aa22d875cc5e18d2da3153310ead2c24a WatchSource:0}: Error finding container b6831c3f2c9e20d74d9c8847890d323aa22d875cc5e18d2da3153310ead2c24a: Status 404 returned error can't find the container with id b6831c3f2c9e20d74d9c8847890d323aa22d875cc5e18d2da3153310ead2c24a
Oct 30 21:11:49.503421 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:49.503387    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b7c30e_9edc_43f6_ba65_279d40a3267e.slice/crio-731e40fc0bbacca627cc99edeba47227c1817c2834bc4ac6d75ed734d933f4d0 WatchSource:0}: Error finding container 731e40fc0bbacca627cc99edeba47227c1817c2834bc4ac6d75ed734d933f4d0: Status 404 returned error can't find the container with id 731e40fc0bbacca627cc99edeba47227c1817c2834bc4ac6d75ed734d933f4d0
Oct 30 21:11:49.551924 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.551855    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zwzcq"]
Oct 30 21:11:49.557669 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:11:49.557642    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b61fec4_1b8d_449c_b251_b48c3306d351.slice/crio-f62229a1a7ae4d01bc19ee2116c7b93d5aecf12c8d00ac2aa80039d1e662dad2 WatchSource:0}: Error finding container f62229a1a7ae4d01bc19ee2116c7b93d5aecf12c8d00ac2aa80039d1e662dad2: Status 404 returned error can't find the container with id f62229a1a7ae4d01bc19ee2116c7b93d5aecf12c8d00ac2aa80039d1e662dad2
Oct 30 21:11:49.646976 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.646942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:11:49.647106 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.646942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:11:49.699896 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.699839    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r8s65\""
Oct 30 21:11:49.699896 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.699839    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:11:49.699896 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.699868    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:11:49.700642 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.699948    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-z8gbk\""
Oct 30 21:11:49.700642 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.699957    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:11:49.836001 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.835909    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjfxt" event={"ID":"b8b7c30e-9edc-43f6-ba65-279d40a3267e","Type":"ContainerStarted","Data":"731e40fc0bbacca627cc99edeba47227c1817c2834bc4ac6d75ed734d933f4d0"}
Oct 30 21:11:49.837451 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.837418    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwzcq" event={"ID":"6b61fec4-1b8d-449c-b251-b48c3306d351","Type":"ContainerStarted","Data":"88195721cd8acaedeaa70e1a4a4c7a63c1fe622a7bf9acde931a952d0c317041"}
Oct 30 21:11:49.837582 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.837462    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwzcq" event={"ID":"6b61fec4-1b8d-449c-b251-b48c3306d351","Type":"ContainerStarted","Data":"f62229a1a7ae4d01bc19ee2116c7b93d5aecf12c8d00ac2aa80039d1e662dad2"}
Oct 30 21:11:49.838548 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:49.838528    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fvn9q" event={"ID":"2238e566-a39a-47ff-8f14-a27a404bc255","Type":"ContainerStarted","Data":"b6831c3f2c9e20d74d9c8847890d323aa22d875cc5e18d2da3153310ead2c24a"}
Oct 30 21:11:51.848175 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:51.847981    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwzcq" event={"ID":"6b61fec4-1b8d-449c-b251-b48c3306d351","Type":"ContainerStarted","Data":"6a6519c6ff0012659c106caa5018d0fc99cc7a4e1238906a5f600a53e46f4fce"}
Oct 30 21:11:52.852855 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:52.852769    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fvn9q" event={"ID":"2238e566-a39a-47ff-8f14-a27a404bc255","Type":"ContainerStarted","Data":"f1aeef27f73381bb1cde9d67834709fc78dcc79d84d1b44edc9a5078ee59c43f"}
Oct 30 21:11:52.855102 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:52.855070    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjfxt" event={"ID":"b8b7c30e-9edc-43f6-ba65-279d40a3267e","Type":"ContainerStarted","Data":"525d389d6bd6d56950110a62c65bb4de15410e533e659fa2ef74b72375b05b0c"}
Oct 30 21:11:52.855232 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:52.855126    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjfxt" event={"ID":"b8b7c30e-9edc-43f6-ba65-279d40a3267e","Type":"ContainerStarted","Data":"41594616970feb2dbf75f9e30228c1417e5b6caf04230a6cf09fa8968121a13e"}
Oct 30 21:11:52.855232 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:52.855162    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:11:52.969229 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:52.969154    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fvn9q" podStartSLOduration=1.882299632 podStartE2EDuration="4.969136546s" totalImagesPullingTime="3.086836914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:52.968829068 +0000 UTC m=+45.764613756" watchObservedRunningTime="2026-10-30 21:11:52.969136546 +0000 UTC m=+45.764921234"
Oct 30 21:11:53.085060 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:53.085003    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-sjfxt" podStartSLOduration=2.833981754 podStartE2EDuration="5.084989357s" totalImagesPullingTime="2.251007603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:53.08449486 +0000 UTC m=+45.880279545" watchObservedRunningTime="2026-10-30 21:11:53.084989357 +0000 UTC m=+45.880774042"
Oct 30 21:11:53.859365 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:53.859310    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwzcq" event={"ID":"6b61fec4-1b8d-449c-b251-b48c3306d351","Type":"ContainerStarted","Data":"9b997bbcae2e079e839e099fcb6f7de237c04ba110143ac6549751e754c03579"}
Oct 30 21:11:53.977497 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:11:53.977443    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zwzcq" podStartSLOduration=2.758310292 podStartE2EDuration="5.977427874s" totalImagesPullingTime="3.219117582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:11:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:53.976971725 +0000 UTC m=+46.772756412" watchObservedRunningTime="2026-10-30 21:11:53.977427874 +0000 UTC m=+46.773212560"
Oct 30 21:12:02.861598 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:02.861401    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:04.640419 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.640387    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"]
Oct 30 21:12:04.642290 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.642271    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:04.744518 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.744478    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 21:12:04.744712 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.744553    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vpcr6\""
Oct 30 21:12:04.745510 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.745493    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"]
Oct 30 21:12:04.755939 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.755912    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7535ee53-f698-47f2-83fa-e0694379654a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-s47fz\" (UID: \"7535ee53-f698-47f2-83fa-e0694379654a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:04.856857 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.856822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7535ee53-f698-47f2-83fa-e0694379654a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-s47fz\" (UID: \"7535ee53-f698-47f2-83fa-e0694379654a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:04.861159 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.861127    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7535ee53-f698-47f2-83fa-e0694379654a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-s47fz\" (UID: \"7535ee53-f698-47f2-83fa-e0694379654a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:04.951639 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:04.951602    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:05.086483 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.086444    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-b2mvs"]
Oct 30 21:12:05.088581 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.088559    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:05.140827 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.140786    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:12:05.140971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.140860    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:12:05.140971 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.140890    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-6sttw\""
Oct 30 21:12:05.191372 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.191343    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-b2mvs"]
Oct 30 21:12:05.260055 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.259958    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68rr5\" (UniqueName: \"kubernetes.io/projected/1daa2700-153a-4fed-8434-669dd9b021bb-kube-api-access-68rr5\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:05.260055 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.260046    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1daa2700-153a-4fed-8434-669dd9b021bb-tmp\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:05.330657 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.330446    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"]
Oct 30 21:12:05.334544 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:05.334513    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7535ee53_f698_47f2_83fa_e0694379654a.slice/crio-d3fa2a371f2d6036f543276331a4fb158e2aee350eca63a40a89e18a0747346b WatchSource:0}: Error finding container d3fa2a371f2d6036f543276331a4fb158e2aee350eca63a40a89e18a0747346b: Status 404 returned error can't find the container with id d3fa2a371f2d6036f543276331a4fb158e2aee350eca63a40a89e18a0747346b
Oct 30 21:12:05.360569 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.360528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1daa2700-153a-4fed-8434-669dd9b021bb-tmp\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:05.360569 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.360569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68rr5\" (UniqueName: \"kubernetes.io/projected/1daa2700-153a-4fed-8434-669dd9b021bb-kube-api-access-68rr5\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:05.360927 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.360906    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1daa2700-153a-4fed-8434-669dd9b021bb-tmp\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:05.417823 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.417793    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68rr5\" (UniqueName: \"kubernetes.io/projected/1daa2700-153a-4fed-8434-669dd9b021bb-kube-api-access-68rr5\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:05.698378 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.698345    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:05.817658 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.817626    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:05.887518 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:05.887485    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz" event={"ID":"7535ee53-f698-47f2-83fa-e0694379654a","Type":"ContainerStarted","Data":"d3fa2a371f2d6036f543276331a4fb158e2aee350eca63a40a89e18a0747346b"}
Oct 30 21:12:06.157415 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:06.157344    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-b2mvs"]
Oct 30 21:12:06.160462 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:06.160425    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1daa2700_153a_4fed_8434_669dd9b021bb.slice/crio-8606df236e3ab41a1bcfe7bc2f8ed734cb041b463d81cf938f901d7ed960f316 WatchSource:0}: Error finding container 8606df236e3ab41a1bcfe7bc2f8ed734cb041b463d81cf938f901d7ed960f316: Status 404 returned error can't find the container with id 8606df236e3ab41a1bcfe7bc2f8ed734cb041b463d81cf938f901d7ed960f316
Oct 30 21:12:06.896619 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:06.896578    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz" event={"ID":"7535ee53-f698-47f2-83fa-e0694379654a","Type":"ContainerStarted","Data":"6c8f4f80c93a02a67fe146c9de355f5bdfab1d46a6a8a6aa35f4e47de960acc5"}
Oct 30 21:12:06.897106 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:06.896777    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:06.898014 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:06.897989    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-b2mvs" event={"ID":"1daa2700-153a-4fed-8434-669dd9b021bb","Type":"ContainerStarted","Data":"8606df236e3ab41a1bcfe7bc2f8ed734cb041b463d81cf938f901d7ed960f316"}
Oct 30 21:12:06.901759 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:06.901743    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:07.013268 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:07.013146    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz" podStartSLOduration=1.607495012 podStartE2EDuration="3.013131981s" totalImagesPullingTime="1.405636969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:07.011862229 +0000 UTC m=+59.807646915" watchObservedRunningTime="2026-10-30 21:12:07.013131981 +0000 UTC m=+59.808916665"
Oct 30 21:12:08.265756 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.265720    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"]
Oct 30 21:12:08.268186 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.268160    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.370319 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.370272    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 21:12:08.370478 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.370355    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xrxgx\""
Oct 30 21:12:08.370746 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.370709    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:12:08.370746 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.370724    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 21:12:08.370938 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.370798    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:12:08.370993 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.370957    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:12:08.385150 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.385123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.385306 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.385177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjbpj\" (UniqueName: \"kubernetes.io/projected/55bed0f4-2cfe-4624-90b8-2f56b728d578-kube-api-access-kjbpj\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.385306 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.385288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55bed0f4-2cfe-4624-90b8-2f56b728d578-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.385413 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.385362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.421197 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.421162    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"]
Oct 30 21:12:08.486770 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.486728    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjbpj\" (UniqueName: \"kubernetes.io/projected/55bed0f4-2cfe-4624-90b8-2f56b728d578-kube-api-access-kjbpj\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.486930 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.486788    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55bed0f4-2cfe-4624-90b8-2f56b728d578-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.486930 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.486844    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.486930 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.486887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.487636 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.487617    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55bed0f4-2cfe-4624-90b8-2f56b728d578-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.490163 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.490129    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.490679 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.490653    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.544622 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.544553    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjbpj\" (UniqueName: \"kubernetes.io/projected/55bed0f4-2cfe-4624-90b8-2f56b728d578-kube-api-access-kjbpj\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.580331 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.580298    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:08.959993 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:08.959953    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"]
Oct 30 21:12:08.993910 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:08.993880    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55bed0f4_2cfe_4624_90b8_2f56b728d578.slice/crio-cc186bbec82ab8f459d5782feb8f03feff5eb8f492b065a5b9163e9f554ff811 WatchSource:0}: Error finding container cc186bbec82ab8f459d5782feb8f03feff5eb8f492b065a5b9163e9f554ff811: Status 404 returned error can't find the container with id cc186bbec82ab8f459d5782feb8f03feff5eb8f492b065a5b9163e9f554ff811
Oct 30 21:12:09.911258 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:09.911222    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr" event={"ID":"55bed0f4-2cfe-4624-90b8-2f56b728d578","Type":"ContainerStarted","Data":"cc186bbec82ab8f459d5782feb8f03feff5eb8f492b065a5b9163e9f554ff811"}
Oct 30 21:12:10.917280 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:10.917240    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr" event={"ID":"55bed0f4-2cfe-4624-90b8-2f56b728d578","Type":"ContainerStarted","Data":"562bc7273baf4df2dda4490879c481029c7657fdc87575823af3a997a40f6d70"}
Oct 30 21:12:10.917769 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:10.917310    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr" event={"ID":"55bed0f4-2cfe-4624-90b8-2f56b728d578","Type":"ContainerStarted","Data":"bea74c949d2f9199f267184eb74e8f0c5c7afc80e10427ae22eef7b42ecf48ac"}
Oct 30 21:12:11.059701 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:11.059595    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr" podStartSLOduration=1.330426019 podStartE2EDuration="3.059580923s" totalImagesPullingTime="1.729154904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:11.058395751 +0000 UTC m=+63.854180436" watchObservedRunningTime="2026-10-30 21:12:11.059580923 +0000 UTC m=+63.855365607"
Oct 30 21:12:12.621811 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.621773    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:12.621811 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.621811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:12:12.722850 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.722784    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:12:12.722850 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.722835    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:12:12.723074 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.722909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:12:12.735551 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.735517    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:12.735682 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.735573    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:12:12.774770 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.774734    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:12:12.834774 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.834734    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:12:12.843605 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.843576    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r8s65\""
Oct 30 21:12:12.843605 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.843592    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tklwz\""
Oct 30 21:12:12.847364 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.847341    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:12:12.852958 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.852936    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:12:12.852958 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:12.852954    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:13.160424 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.160392    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-z8gbk\""
Oct 30 21:12:13.169534 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.169495    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:12:13.239377 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.239340    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nbcn2"]
Oct 30 21:12:13.248725 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:13.248697    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2359dc_5156_4bd5_8e7f_1bce48d948e6.slice/crio-b176117b104c6ba75d453da4c5f021bd3fca01295c2ee5af4a5734afce77905d WatchSource:0}: Error finding container b176117b104c6ba75d453da4c5f021bd3fca01295c2ee5af4a5734afce77905d: Status 404 returned error can't find the container with id b176117b104c6ba75d453da4c5f021bd3fca01295c2ee5af4a5734afce77905d
Oct 30 21:12:13.290801 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.290770    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r59mn"]
Oct 30 21:12:13.294306 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:13.294276    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0492a73c_e41b_46c7_ad57_1f8220dfcf3a.slice/crio-5a5a34150e3e7a1212eb3d96c6573ba1e59c5cfdc8f697c2cd859e39023b6340 WatchSource:0}: Error finding container 5a5a34150e3e7a1212eb3d96c6573ba1e59c5cfdc8f697c2cd859e39023b6340: Status 404 returned error can't find the container with id 5a5a34150e3e7a1212eb3d96c6573ba1e59c5cfdc8f697c2cd859e39023b6340
Oct 30 21:12:13.549496 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.549417    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c65g6"]
Oct 30 21:12:13.552389 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:13.552366    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5786b3_e466_4fc8_99c6_bc2876267401.slice/crio-e87150dd997f161a0deb79703155a9b007c0cb011f1b3b7933f07df4f6168b10 WatchSource:0}: Error finding container e87150dd997f161a0deb79703155a9b007c0cb011f1b3b7933f07df4f6168b10: Status 404 returned error can't find the container with id e87150dd997f161a0deb79703155a9b007c0cb011f1b3b7933f07df4f6168b10
Oct 30 21:12:13.845869 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.845796    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"]
Oct 30 21:12:13.848876 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.848855    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:13.900909 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.900814    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:12:13.900909 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.900818    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:12:13.901135 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.900952    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vwtq2\""
Oct 30 21:12:13.906616 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.906582    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"]
Oct 30 21:12:13.931785 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.931756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:13.931955 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.931809    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92966fe2-fe0f-494f-9bc8-87000dd086c9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:13.931955 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.931841    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:13.931955 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.931869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzngr\" (UniqueName: \"kubernetes.io/projected/92966fe2-fe0f-494f-9bc8-87000dd086c9-kube-api-access-vzngr\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:13.932981 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.932954    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c65g6" event={"ID":"4a5786b3-e466-4fc8-99c6-bc2876267401","Type":"ContainerStarted","Data":"e87150dd997f161a0deb79703155a9b007c0cb011f1b3b7933f07df4f6168b10"}
Oct 30 21:12:13.934301 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.934276    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r59mn" event={"ID":"0492a73c-e41b-46c7-ad57-1f8220dfcf3a","Type":"ContainerStarted","Data":"5a5a34150e3e7a1212eb3d96c6573ba1e59c5cfdc8f697c2cd859e39023b6340"}
Oct 30 21:12:13.935524 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.935492    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbcn2" event={"ID":"fe2359dc-5156-4bd5-8e7f-1bce48d948e6","Type":"ContainerStarted","Data":"b176117b104c6ba75d453da4c5f021bd3fca01295c2ee5af4a5734afce77905d"}
Oct 30 21:12:13.955238 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.954772    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"]
Oct 30 21:12:13.956764 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.956749    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:13.962426 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.962403    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5gr7z"]
Oct 30 21:12:13.964930 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:13.964911    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.009002 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.008973    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:12:14.009175 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.009015    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rc6nc\""
Oct 30 21:12:14.009175 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.009015    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:12:14.009175 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.009050    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:12:14.018099 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.018071    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"]
Oct 30 21:12:14.032269 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032233    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-sys\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.032414 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032314    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.032414 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032356    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzngr\" (UniqueName: \"kubernetes.io/projected/92966fe2-fe0f-494f-9bc8-87000dd086c9-kube-api-access-vzngr\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.032414 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032383    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-metrics-client-ca\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.032414 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032411    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swj72\" (UniqueName: \"kubernetes.io/projected/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-api-access-swj72\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.032609 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032443    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.032691 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032669    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.032746 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032718    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92966fe2-fe0f-494f-9bc8-87000dd086c9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.032785 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032767    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-wtmp\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.032870 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032855    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-tls\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.032925 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032885    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.032925 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.032918    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.033612 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.033441    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0eb207a7-1d34-4edf-8e0a-488b4363e175-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.033612 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.033467    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92966fe2-fe0f-494f-9bc8-87000dd086c9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.033612 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.033500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.033612 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.033528    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-root\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.033612 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.033555    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6kcg\" (UniqueName: \"kubernetes.io/projected/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-kube-api-access-n6kcg\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.033612 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.033578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.033925 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.033633    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-textfile\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.033925 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.033745    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.035394 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.035373    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.035629 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.035604    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.049446 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.049412    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:12:14.097599 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.097532    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzngr\" (UniqueName: \"kubernetes.io/projected/92966fe2-fe0f-494f-9bc8-87000dd086c9-kube-api-access-vzngr\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.109302 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.109271    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wxcc9\""
Oct 30 21:12:14.109468 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.109305    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:12:14.109468 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.109365    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:12:14.134747 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-sys\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.134913 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134763    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.134913 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-metrics-client-ca\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.134913 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134813    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swj72\" (UniqueName: \"kubernetes.io/projected/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-api-access-swj72\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.134913 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.134913 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134849    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-sys\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.134913 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134866    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-wtmp\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-tls\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.135172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.134983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0eb207a7-1d34-4edf-8e0a-488b4363e175-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.135172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135024    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135048    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-root\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135065    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-wtmp\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6kcg\" (UniqueName: \"kubernetes.io/projected/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-kube-api-access-n6kcg\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135101    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.135430 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-metrics-client-ca\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135461 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135438    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135492 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135467    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-textfile\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135523 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135494    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-root\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.135877 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.135877 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135803    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0eb207a7-1d34-4edf-8e0a-488b4363e175-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.135877 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.135840    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-textfile\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.136196 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.136170    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.136298 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.136239    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.138199 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.138178    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.138934 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.138909    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-tls\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.139034 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.138915    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.139384 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.139359    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.164276 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.164244    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:14.194497 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.194441    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6kcg\" (UniqueName: \"kubernetes.io/projected/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-kube-api-access-n6kcg\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.196439 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.196414    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swj72\" (UniqueName: \"kubernetes.io/projected/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-api-access-swj72\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.269297 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.269104    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:14.276412 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.276380    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:14.291540 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:14.291440    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf3b1a7_af00_46ce_a1fc_70b02516a53f.slice/crio-bb27ebef79a610f4e6e4143ebc4253f4a644cfd67a8799f63d18b90f9bbab747 WatchSource:0}: Error finding container bb27ebef79a610f4e6e4143ebc4253f4a644cfd67a8799f63d18b90f9bbab747: Status 404 returned error can't find the container with id bb27ebef79a610f4e6e4143ebc4253f4a644cfd67a8799f63d18b90f9bbab747
Oct 30 21:12:14.553861 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.553817    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"]
Oct 30 21:12:14.559576 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:14.559544    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92966fe2_fe0f_494f_9bc8_87000dd086c9.slice/crio-0983ffbaeae93fc95074f36179d59011469de179bc13013c28324019604ceaa8 WatchSource:0}: Error finding container 0983ffbaeae93fc95074f36179d59011469de179bc13013c28324019604ceaa8: Status 404 returned error can't find the container with id 0983ffbaeae93fc95074f36179d59011469de179bc13013c28324019604ceaa8
Oct 30 21:12:14.668395 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.668070    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"]
Oct 30 21:12:14.680891 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:14.680744    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb207a7_1d34_4edf_8e0a_488b4363e175.slice/crio-cb1e06307a77553a8b67e40b8f6b32f03e00357c369580f4e9f635e90498f4ff WatchSource:0}: Error finding container cb1e06307a77553a8b67e40b8f6b32f03e00357c369580f4e9f635e90498f4ff: Status 404 returned error can't find the container with id cb1e06307a77553a8b67e40b8f6b32f03e00357c369580f4e9f635e90498f4ff
Oct 30 21:12:14.943456 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.943343    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj" event={"ID":"0eb207a7-1d34-4edf-8e0a-488b4363e175","Type":"ContainerStarted","Data":"cb1e06307a77553a8b67e40b8f6b32f03e00357c369580f4e9f635e90498f4ff"}
Oct 30 21:12:14.945697 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.945628    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gr7z" event={"ID":"9cf3b1a7-af00-46ce-a1fc-70b02516a53f","Type":"ContainerStarted","Data":"bb27ebef79a610f4e6e4143ebc4253f4a644cfd67a8799f63d18b90f9bbab747"}
Oct 30 21:12:14.948479 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.948339    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p" event={"ID":"92966fe2-fe0f-494f-9bc8-87000dd086c9","Type":"ContainerStarted","Data":"baa3e9e3c9e1ce8f095c1c09aaa4be9a45d029a02a8c351ff140ff09d137fe68"}
Oct 30 21:12:14.948479 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.948395    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p" event={"ID":"92966fe2-fe0f-494f-9bc8-87000dd086c9","Type":"ContainerStarted","Data":"7e3c721659bf07ec8268f413dfd86225a9f7135936d9a783a64476b44588cddb"}
Oct 30 21:12:14.948479 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:14.948422    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p" event={"ID":"92966fe2-fe0f-494f-9bc8-87000dd086c9","Type":"ContainerStarted","Data":"0983ffbaeae93fc95074f36179d59011469de179bc13013c28324019604ceaa8"}
Oct 30 21:12:15.957797 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:15.956878    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbcn2" event={"ID":"fe2359dc-5156-4bd5-8e7f-1bce48d948e6","Type":"ContainerStarted","Data":"6aec5ec115e3af6ab4c03a332ff007ab0160736e06bbbb281357f7c5a85943d4"}
Oct 30 21:12:15.957797 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:15.956946    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbcn2" event={"ID":"fe2359dc-5156-4bd5-8e7f-1bce48d948e6","Type":"ContainerStarted","Data":"56a5d9ce127ffde3390bc3fed58a4a518f53fa3c94b34d6caad8b745f7826576"}
Oct 30 21:12:16.288874 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.288808    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nbcn2" podStartSLOduration=67.42530214 podStartE2EDuration="1m9.288786017s" totalImagesPullingTime="1.863483877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:16.073091312 +0000 UTC m=+68.868875998" watchObservedRunningTime="2026-10-30 21:12:16.288786017 +0000 UTC m=+69.084570704"
Oct 30 21:12:16.290013 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.289890    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:12:16.293171 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.293142    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354470    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354513    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354543    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354571    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354646    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354678    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354700    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354734    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354764    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.355073 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.354842    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgck7\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.397311 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.397268    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:12:16.426643 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.426119    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:12:16.446190 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.446156    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:12:16.448319 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.448290    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:12:16.448480 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.448397    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:12:16.448480 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.448462    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:12:16.448559 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.448509    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:12:16.448959 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.448610    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ljsb2\""
Oct 30 21:12:16.449061 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.448892    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:12:16.449061 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.448921    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:12:16.449061 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.449045    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.455729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.455776    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.455816    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.455845    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.455902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.455925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgck7\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.455962    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.455984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.456013    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.456047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.456074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.456108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.457255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.456135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.461450 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.458874    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.461450 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.460882    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.461450 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.461329    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.461761 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.461543    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.462118 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.462062    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.462348 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.462323    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.466842 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.466583    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.467380 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.467238    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.468785 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.468757    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.471995 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.471941    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.472578 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.472530    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.475557 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.475510    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.502629 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.500765    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-646bb56695-d6qr8"]
Oct 30 21:12:16.504668 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.504612    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.513710 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.513645    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgck7\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.557161 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557124    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-tmp\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.557330 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557185    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc2mr\" (UniqueName: \"kubernetes.io/projected/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-kube-api-access-sc2mr\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.557330 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.557330 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.557416 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557328    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-grpc-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.557416 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557361    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.557416 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-metrics-client-ca\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.557541 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.557541 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.557465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.558703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.558365    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:12:16.558703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.558488    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:12:16.558703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.558506    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4fqz7\""
Oct 30 21:12:16.558703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.558575    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:12:16.558703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.558609    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:12:16.558703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.558647    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:12:16.559011 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.558833    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a4l5rnn1lts2h\""
Oct 30 21:12:16.566793 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.566633    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-646bb56695-d6qr8"]
Oct 30 21:12:16.619276 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.619244    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:16.652730 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.651499    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-75579f5f75-plqsn"]
Oct 30 21:12:16.655392 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.654283    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.666955 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.666929    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-tmp\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.667124 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667105    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx5b5\" (UniqueName: \"kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.667247 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667231    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.667362 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667347    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc2mr\" (UniqueName: \"kubernetes.io/projected/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-kube-api-access-sc2mr\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.667471 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667458    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.667609 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667593    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.667715 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667700    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.667818 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667804    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.667924 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667890    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.668039 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.668025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-grpc-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.668155 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.668141    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.668279 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.668264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.668389 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.668372    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-metrics-client-ca\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.668502 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.668487    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.668995 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.668594    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.668995 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.668639    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.669421 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.667724    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-tmp\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.672430 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.671694    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.672430 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.672264    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-metrics-client-ca\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.673670 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.673596    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-grpc-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.674665 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.674636    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.684309 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.684229    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.684923 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.684873    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.690591 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.690545    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.715989 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.714593    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4k42l\""
Oct 30 21:12:16.715989 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.714729    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:12:16.715989 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.714866    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:12:16.715989 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.715114    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:12:16.715989 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.715227    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:12:16.721362 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.716468    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:12:16.723058 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.722941    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75579f5f75-plqsn"]
Oct 30 21:12:16.732249 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.732162    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc2mr\" (UniqueName: \"kubernetes.io/projected/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-kube-api-access-sc2mr\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.769358 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.769322    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.769358 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.769361    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.769604 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.769378    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.769604 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.769410    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.769604 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.769441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.769604 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.769468    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx5b5\" (UniqueName: \"kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.769604 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.769485    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.771146 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.770080    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.771146 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.770889    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.771384 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.771359    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.771547 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.771524    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.774743 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.774715    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.776705 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.776660    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.827681 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.827640    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:16.832341 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.832281    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx5b5\" (UniqueName: \"kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:16.970546 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.970277    2574 generic.go:356] "Generic (PLEG): container finished" podID="9cf3b1a7-af00-46ce-a1fc-70b02516a53f" containerID="7328fb709033dbf0cdfa174de2a1d812e5094047a7104e9138bed1465f531146" exitCode=0
Oct 30 21:12:16.971919 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.971381    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gr7z" event={"ID":"9cf3b1a7-af00-46ce-a1fc-70b02516a53f","Type":"ContainerDied","Data":"7328fb709033dbf0cdfa174de2a1d812e5094047a7104e9138bed1465f531146"}
Oct 30 21:12:16.978580 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:16.978313    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:17.090930 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.090874    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:12:17.099292 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:17.099259    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2f2767_292c_45d1_a573_fb2749f866fe.slice/crio-2dea41af0a258a3b1a12868035a38ac48c13b4da0a66207a22e2e2058183566f WatchSource:0}: Error finding container 2dea41af0a258a3b1a12868035a38ac48c13b4da0a66207a22e2e2058183566f: Status 404 returned error can't find the container with id 2dea41af0a258a3b1a12868035a38ac48c13b4da0a66207a22e2e2058183566f
Oct 30 21:12:17.227262 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.227158    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-646bb56695-d6qr8"]
Oct 30 21:12:17.231317 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:17.231286    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75fd1f7_4fef_489b_ab98_45bbaf194bdf.slice/crio-340a3c8fdb92d2bbf407cf693effa8ac046ac56ebcde9ce08714e4cbf2d6942a WatchSource:0}: Error finding container 340a3c8fdb92d2bbf407cf693effa8ac046ac56ebcde9ce08714e4cbf2d6942a: Status 404 returned error can't find the container with id 340a3c8fdb92d2bbf407cf693effa8ac046ac56ebcde9ce08714e4cbf2d6942a
Oct 30 21:12:17.370599 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.370567    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75579f5f75-plqsn"]
Oct 30 21:12:17.375472 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:17.375447    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2007f2be_74e2_4db5_ba7e_c3e2596777d6.slice/crio-d60eaa159a472139d1e983915b0ee2aaaa5faff5a84b30404dbc2d8457c12066 WatchSource:0}: Error finding container d60eaa159a472139d1e983915b0ee2aaaa5faff5a84b30404dbc2d8457c12066: Status 404 returned error can't find the container with id d60eaa159a472139d1e983915b0ee2aaaa5faff5a84b30404dbc2d8457c12066
Oct 30 21:12:17.983122 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.982956    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gr7z" event={"ID":"9cf3b1a7-af00-46ce-a1fc-70b02516a53f","Type":"ContainerStarted","Data":"2d48d577c764f04878f02fa6426a0b77fd4d12314c7df87a80b5b6cfbdc2cf29"}
Oct 30 21:12:17.983122 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.983078    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gr7z" event={"ID":"9cf3b1a7-af00-46ce-a1fc-70b02516a53f","Type":"ContainerStarted","Data":"05930bd0947285577393d6ec0b3c7f24f63d4a910a9018c4ae570b6d8e19d5b2"}
Oct 30 21:12:17.987017 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.986576    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p" event={"ID":"92966fe2-fe0f-494f-9bc8-87000dd086c9","Type":"ContainerStarted","Data":"822091be12f458b5e3f8bf4ace45cee81a7525ade5dbda51d6bbda780e725d45"}
Oct 30 21:12:17.989919 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.989884    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj" event={"ID":"0eb207a7-1d34-4edf-8e0a-488b4363e175","Type":"ContainerStarted","Data":"a07cd9a8170417ce131bda5b69250ff3a29447715f75980c1b616360fed0f5bd"}
Oct 30 21:12:17.990069 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.989951    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj" event={"ID":"0eb207a7-1d34-4edf-8e0a-488b4363e175","Type":"ContainerStarted","Data":"8f9d63492365066c68db6ff8dba9300167d03493e6f53a48a25249bf23568ba9"}
Oct 30 21:12:17.990069 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.989978    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj" event={"ID":"0eb207a7-1d34-4edf-8e0a-488b4363e175","Type":"ContainerStarted","Data":"444fc9a6000e7ebd09c58a142de556c7af81f9dc47b7a5ac93be5d647ddb5852"}
Oct 30 21:12:17.992469 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.992438    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8" event={"ID":"e75fd1f7-4fef-489b-ab98-45bbaf194bdf","Type":"ContainerStarted","Data":"340a3c8fdb92d2bbf407cf693effa8ac046ac56ebcde9ce08714e4cbf2d6942a"}
Oct 30 21:12:17.994121 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.994078    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerStarted","Data":"2dea41af0a258a3b1a12868035a38ac48c13b4da0a66207a22e2e2058183566f"}
Oct 30 21:12:17.995740 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:17.995693    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75579f5f75-plqsn" event={"ID":"2007f2be-74e2-4db5-ba7e-c3e2596777d6","Type":"ContainerStarted","Data":"d60eaa159a472139d1e983915b0ee2aaaa5faff5a84b30404dbc2d8457c12066"}
Oct 30 21:12:18.102550 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.101882    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5gr7z" podStartSLOduration=3.172836699 podStartE2EDuration="5.101863435s" totalImagesPullingTime="1.831961621s" totalInitContainerRuntime="97.065115ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:18.101059832 +0000 UTC m=+70.896844518" watchObservedRunningTime="2026-10-30 21:12:18.101863435 +0000 UTC m=+70.897648121"
Oct 30 21:12:18.222382 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.221926    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj" podStartSLOduration=2.430920142 podStartE2EDuration="5.221906074s" totalImagesPullingTime="2.790985932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:18.221497611 +0000 UTC m=+71.017282296" watchObservedRunningTime="2026-10-30 21:12:18.221906074 +0000 UTC m=+71.017690763"
Oct 30 21:12:18.342772 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.342650    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p" podStartSLOduration=3.18382083 podStartE2EDuration="5.342628489s" totalImagesPullingTime="2.158807659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:18.34083832 +0000 UTC m=+71.136623005" watchObservedRunningTime="2026-10-30 21:12:18.342628489 +0000 UTC m=+71.138413176"
Oct 30 21:12:18.645952 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.645869    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"]
Oct 30 21:12:18.649012 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.648989    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:18.691742 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.690589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlk7x\" (UniqueName: \"kubernetes.io/projected/fe7b8332-a94f-40a0-b4b5-5880f10b571a-kube-api-access-hlk7x\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:18.691742 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.690726    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe7b8332-a94f-40a0-b4b5-5880f10b571a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:18.706172 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.706110    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"]
Oct 30 21:12:18.730839 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.730794    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:12:18.731107 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.731083    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vcb5m\""
Oct 30 21:12:18.791431 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.791392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlk7x\" (UniqueName: \"kubernetes.io/projected/fe7b8332-a94f-40a0-b4b5-5880f10b571a-kube-api-access-hlk7x\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:18.791617 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.791497    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe7b8332-a94f-40a0-b4b5-5880f10b571a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:18.797790 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.797758    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe7b8332-a94f-40a0-b4b5-5880f10b571a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:18.850146 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.850071    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlk7x\" (UniqueName: \"kubernetes.io/projected/fe7b8332-a94f-40a0-b4b5-5880f10b571a-kube-api-access-hlk7x\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:18.886606 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.886568    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7c86794bcf-ck4wn"]
Oct 30 21:12:18.889103 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.889076    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:18.941769 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.941743    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:12:18.941951 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.941810    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 21:12:18.949108 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.949080    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c86794bcf-ck4wn"]
Oct 30 21:12:18.964362 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.964332    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:18.975235 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.975190    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 21:12:18.975365 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.975342    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 21:12:18.975443 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.975415    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1fddjiuem6drv\""
Oct 30 21:12:18.975503 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.975470    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pmmk6\""
Oct 30 21:12:18.993182 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.993153    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-tls\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:18.993599 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.993236    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fdf5f020-799f-4fc6-86c2-27b76e7bb567-audit-log\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:18.993599 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.993280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-client-certs\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:18.993599 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.993319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8vsb\" (UniqueName: \"kubernetes.io/projected/fdf5f020-799f-4fc6-86c2-27b76e7bb567-kube-api-access-v8vsb\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:18.993599 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.993353    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:18.993599 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.993392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-metrics-server-audit-profiles\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:18.993599 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:18.993423    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-client-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.094176    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-tls\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.094273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fdf5f020-799f-4fc6-86c2-27b76e7bb567-audit-log\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.094358    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-client-certs\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.094401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8vsb\" (UniqueName: \"kubernetes.io/projected/fdf5f020-799f-4fc6-86c2-27b76e7bb567-kube-api-access-v8vsb\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.094431    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.094531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-metrics-server-audit-profiles\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.094571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-client-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.096661    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.098045 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.097108    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fdf5f020-799f-4fc6-86c2-27b76e7bb567-audit-log\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.101065 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.101022    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-metrics-server-audit-profiles\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.101855 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.101763    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-client-certs\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.102422 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.102380    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-tls\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.102900 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.102861    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-client-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.156410 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.156334    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8vsb\" (UniqueName: \"kubernetes.io/projected/fdf5f020-799f-4fc6-86c2-27b76e7bb567-kube-api-access-v8vsb\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.177431 ip-10-0-132-134 kubenswrapper[2574]: E1030 21:12:19.177392    2574 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:12:19.205822 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.205759    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:19.368353 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.367455    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"]
Oct 30 21:12:19.374597 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:19.374566    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe7b8332_a94f_40a0_b4b5_5880f10b571a.slice/crio-d226dc6231c66ddbe1c414b394439e2e2ffa5227b4ed96858749c92d531b0c5a WatchSource:0}: Error finding container d226dc6231c66ddbe1c414b394439e2e2ffa5227b4ed96858749c92d531b0c5a: Status 404 returned error can't find the container with id d226dc6231c66ddbe1c414b394439e2e2ffa5227b4ed96858749c92d531b0c5a
Oct 30 21:12:19.600760 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.600726    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78659679-f77rg"]
Oct 30 21:12:19.603964 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.603943    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.616851 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.616563    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c86794bcf-ck4wn"]
Oct 30 21:12:19.624994 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:19.624965    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf5f020_799f_4fc6_86c2_27b76e7bb567.slice/crio-4dcb576a5d7db4d871f30cff4869ae4b04f4b2a68da30d8557f3c8eb09076577 WatchSource:0}: Error finding container 4dcb576a5d7db4d871f30cff4869ae4b04f4b2a68da30d8557f3c8eb09076577: Status 404 returned error can't find the container with id 4dcb576a5d7db4d871f30cff4869ae4b04f4b2a68da30d8557f3c8eb09076577
Oct 30 21:12:19.671255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.669925    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:12:19.671255 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.670324    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:12:19.672390 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.672065    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:12:19.674230 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.672559    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2fnqg\""
Oct 30 21:12:19.674230 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.672737    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:12:19.674230 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.672928    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:12:19.676139 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.676118    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:12:19.679942 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.679910    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78659679-f77rg"]
Oct 30 21:12:19.704320 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.704242    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.704320 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.704314    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl9k5\" (UniqueName: \"kubernetes.io/projected/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-kube-api-access-pl9k5\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.704515 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.704336    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.704515 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.704365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.704515 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.704460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-serving-certs-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.704515 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.704502    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.704703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.704551    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-metrics-client-ca\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.704703 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.704578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-federate-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.806504 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.805755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-metrics-client-ca\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.806504 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.805805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-federate-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.806504 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.805859    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.806504 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.805893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl9k5\" (UniqueName: \"kubernetes.io/projected/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-kube-api-access-pl9k5\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.806504 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.805922    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.806504 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.805952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.806504 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.805986    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-serving-certs-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.806504 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.806016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.808637 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.808006    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-metrics-client-ca\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.809911 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.809882    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-serving-certs-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.810059 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.810038    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.812233 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.811865    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-federate-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.812466 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.812423    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.813493 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.813368    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.814905 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.814863    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.865758 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.865690    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl9k5\" (UniqueName: \"kubernetes.io/projected/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-kube-api-access-pl9k5\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:19.925829 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:19.925543    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:20.017327 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:20.016894    2574 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2" exitCode=0
Oct 30 21:12:20.017327 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:20.017016    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2"}
Oct 30 21:12:20.024258 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:20.023950    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn" event={"ID":"fdf5f020-799f-4fc6-86c2-27b76e7bb567","Type":"ContainerStarted","Data":"4dcb576a5d7db4d871f30cff4869ae4b04f4b2a68da30d8557f3c8eb09076577"}
Oct 30 21:12:20.030887 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:20.030857    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6" event={"ID":"fe7b8332-a94f-40a0-b4b5-5880f10b571a","Type":"ContainerStarted","Data":"d226dc6231c66ddbe1c414b394439e2e2ffa5227b4ed96858749c92d531b0c5a"}
Oct 30 21:12:20.033769 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:20.033743    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c65g6" event={"ID":"4a5786b3-e466-4fc8-99c6-bc2876267401","Type":"ContainerStarted","Data":"e2a55d75936e624f58d7160c7f30af7aa266c42aee303914a2ab2e8a5309be70"}
Oct 30 21:12:20.034376 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:20.034355    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:12:20.259717 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:20.259605    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c65g6" podStartSLOduration=67.364029699 podStartE2EDuration="1m13.259582989s" totalImagesPullingTime="5.89555329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:20.258511339 +0000 UTC m=+73.054296024" watchObservedRunningTime="2026-10-30 21:12:20.259582989 +0000 UTC m=+73.055367675"
Oct 30 21:12:20.393715 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:20.393682    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78659679-f77rg"]
Oct 30 21:12:20.396073 ip-10-0-132-134 kubenswrapper[2574]: W1030 21:12:20.396037    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c7d804a_c2a5_4923_8a9b_ed2803fdd02a.slice/crio-3b894f85b2fe38d45531b3f85ba1527ec36cd0d53445805e280b22c3026e304d WatchSource:0}: Error finding container 3b894f85b2fe38d45531b3f85ba1527ec36cd0d53445805e280b22c3026e304d: Status 404 returned error can't find the container with id 3b894f85b2fe38d45531b3f85ba1527ec36cd0d53445805e280b22c3026e304d
Oct 30 21:12:21.046618 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:21.046579    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78659679-f77rg" event={"ID":"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a","Type":"ContainerStarted","Data":"3b894f85b2fe38d45531b3f85ba1527ec36cd0d53445805e280b22c3026e304d"}
Oct 30 21:12:22.058107 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:22.057946    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8" event={"ID":"e75fd1f7-4fef-489b-ab98-45bbaf194bdf","Type":"ContainerStarted","Data":"9fa39e14e63312aa32937ab1b6953188d7f7471194f1ade229ff51a06762f287"}
Oct 30 21:12:22.058107 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:22.058005    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8" event={"ID":"e75fd1f7-4fef-489b-ab98-45bbaf194bdf","Type":"ContainerStarted","Data":"54871163681ad98342b605b6093738daee593a7571dea4f7455a5f007c895907"}
Oct 30 21:12:22.058107 ip-10-0-132-134 kubenswrapper[2574]: I1030 21:12:22.058031    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8" event={"ID":"e75fd1f7-4fef-489b-ab98-45bbaf194bdf","Type":"ContainerStarted","Data":"4102e498774daf3b25d1a68f32d54a8fe56c561bca71de45c2d1adaa35eda000"}
Oct 30 21:12:24.045064 ip-10-0-132-134 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:12:24.066704 ip-10-0-132-134 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:12:24.066954 ip-10-0-132-134 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:12:24.067237 ip-10-0-132-134 systemd[1]: kubelet.service: Consumed 4.452s CPU time.
Oct 30 21:12:24.078072 ip-10-0-132-134 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.202996    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203105    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203110    5786 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203115    5786 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203119    5786 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203125    5786 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203129    5786 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203133    5786 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203137    5786 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203142    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:12:24.204240 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203146    5786 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203150    5786 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203155    5786 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203159    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203163    5786 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203168    5786 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203172    5786 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203177    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203182    5786 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203193    5786 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203197    5786 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203201    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203221    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203226    5786 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203243    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203249    5786 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203253    5786 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203257    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203262    5786 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:12:24.206125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203266    5786 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203273    5786 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203278    5786 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203283    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203287    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203292    5786 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203296    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203300    5786 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203304    5786 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203308    5786 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203312    5786 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203316    5786 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203321    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203327    5786 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203331    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203339    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203343    5786 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203347    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203353    5786 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:12:24.206631 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203361    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203366    5786 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203370    5786 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203374    5786 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203380    5786 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203384    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203388    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203393    5786 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203397    5786 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203402    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203406    5786 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203410    5786 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203414    5786 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203419    5786 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203423    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203427    5786 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203432    5786 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203436    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203440    5786 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203444    5786 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:12:24.207104 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203448    5786 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203452    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203456    5786 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203460    5786 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203464    5786 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203468    5786 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203472    5786 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203476    5786 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203482    5786 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203486    5786 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203491    5786 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203495    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203502    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203506    5786 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203510    5786 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203515    5786 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203519    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203524    5786 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203528    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203532    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:12:24.207638 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203536    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203540    5786 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203544    5786 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203548    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203552    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203556    5786 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203560    5786 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203564    5786 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203568    5786 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203572    5786 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203576    5786 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203580    5786 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203584    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203588    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203593    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203597    5786 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203601    5786 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.203605    5786 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204941    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:12:24.208130 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204952    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204960    5786 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204967    5786 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204974    5786 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204979    5786 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204983    5786 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204987    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204991    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.204996    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205000    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205005    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205010    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205014    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205018    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205022    5786 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205026    5786 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205031    5786 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205035    5786 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205039    5786 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:12:24.208688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205043    5786 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205049    5786 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205055    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205060    5786 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205064    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205068    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205072    5786 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205077    5786 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205082    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205086    5786 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205090    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205094    5786 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205098    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205102    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205108    5786 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205112    5786 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205117    5786 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205122    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205126    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:12:24.209506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205130    5786 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205134    5786 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205137    5786 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205141    5786 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205146    5786 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205150    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205154    5786 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205157    5786 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205159    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205162    5786 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205164    5786 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205167    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205169    5786 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205172    5786 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205174    5786 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205177    5786 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205180    5786 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205182    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205185    5786 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205188    5786 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:12:24.210387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205192    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205196    5786 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205201    5786 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205239    5786 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205244    5786 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205247    5786 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205250    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205254    5786 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205256    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205259    5786 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205262    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205266    5786 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205268    5786 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205271    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205273    5786 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205276    5786 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205279    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205283    5786 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205285    5786 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:12:24.211263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205289    5786 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205292    5786 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205294    5786 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205297    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205299    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205302    5786 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205304    5786 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205307    5786 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205310    5786 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205312    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205315    5786 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205317    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205320    5786 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205323    5786 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205325    5786 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205328    5786 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205330    5786 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205333    5786 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205335    5786 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205338    5786 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:12:24.212057 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205342    5786 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205344    5786 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205347    5786 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205350    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205354    5786 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205358    5786 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205363    5786 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.205366    5786 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209912    5786 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209930    5786 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209936    5786 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209940    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209950    5786 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209954    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209959    5786 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209963    5786 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209968    5786 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209973    5786 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209978    5786 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209982    5786 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:12:24.213157 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209987    5786 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209991    5786 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.209996    5786 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210001    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210006    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210015    5786 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210020    5786 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210024    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210028    5786 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210032    5786 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210036    5786 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210040    5786 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210045    5786 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210049    5786 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210053    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210060    5786 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210065    5786 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210070    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210080    5786 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210084    5786 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:12:24.214048 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210089    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210097    5786 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210102    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210107    5786 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210111    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210115    5786 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210119    5786 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210123    5786 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210130    5786 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210138    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210148    5786 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210153    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210158    5786 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210163    5786 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210167    5786 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210171    5786 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210176    5786 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210180    5786 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210185    5786 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:12:24.214918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210190    5786 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210194    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210198    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210217    5786 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210228    5786 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210234    5786 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210245    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210249    5786 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210254    5786 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210258    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210262    5786 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210267    5786 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210272    5786 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210276    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210280    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210285    5786 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210295    5786 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210299    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210304    5786 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:12:24.215794 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210310    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210314    5786 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210319    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210323    5786 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210328    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210332    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210337    5786 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210341    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210346    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210356    5786 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210361    5786 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210367    5786 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210371    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210376    5786 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210381    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210385    5786 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210390    5786 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210397    5786 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210407    5786 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:12:24.216698 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210413    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210417    5786 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210428    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210432    5786 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210436    5786 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210441    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210445    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210449    5786 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210453    5786 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210458    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210464    5786 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210468    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210472    5786 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210477    5786 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210482    5786 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210496    5786 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.210504    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210705    5786 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210725    5786 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210735    5786 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210742    5786 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:12:24.217518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210756    5786 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210761    5786 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210769    5786 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210777    5786 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210782    5786 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210788    5786 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210794    5786 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210800    5786 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210810    5786 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210815    5786 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210819    5786 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210824    5786 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210829    5786 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210834    5786 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210843    5786 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210848    5786 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210854    5786 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210864    5786 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210870    5786 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210878    5786 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210889    5786 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210895    5786 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210901    5786 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210906    5786 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:12:24.218320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210911    5786 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210920    5786 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210926    5786 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210931    5786 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210939    5786 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210944    5786 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210949    5786 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210955    5786 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210960    5786 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210965    5786 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210977    5786 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210982    5786 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210987    5786 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210994    5786 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.210999    5786 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211007    5786 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211012    5786 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211017    5786 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211027    5786 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211032    5786 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211037    5786 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211048    5786 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211053    5786 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211058    5786 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211063    5786 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:12:24.219276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211067    5786 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211078    5786 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211083    5786 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211089    5786 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211094    5786 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211101    5786 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211108    5786 flags.go:64] FLAG: --help="false"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211112    5786 flags.go:64] FLAG: --hostname-override="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211118    5786 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211124    5786 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211134    5786 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211140    5786 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211146    5786 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211152    5786 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211157    5786 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211162    5786 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211167    5786 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211172    5786 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211177    5786 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211186    5786 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211191    5786 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211196    5786 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211274    5786 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:12:24.220282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211286    5786 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211292    5786 flags.go:64] FLAG: --lock-file=""
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211340    5786 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211346    5786 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211358    5786 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211369    5786 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211375    5786 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211380    5786 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211400    5786 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211405    5786 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211431    5786 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211437    5786 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211448    5786 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211459    5786 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211465    5786 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211472    5786 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211480    5786 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211547    5786 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211554    5786 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211588    5786 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211599    5786 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211634    5786 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211642    5786 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211681    5786 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211687    5786 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:12:24.220931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211692    5786 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211697    5786 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211703    5786 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211715    5786 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211721    5786 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211749    5786 flags.go:64] FLAG: --port="10250"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211756    5786 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211762    5786 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0c6b1289c15dcabf0"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211768    5786 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211774    5786 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211779    5786 flags.go:64] FLAG: --register-node="true"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211789    5786 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211796    5786 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211801    5786 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211843    5786 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211852    5786 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211859    5786 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211865    5786 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211871    5786 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211887    5786 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211892    5786 flags.go:64] FLAG: --runonce="false"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211897    5786 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211903    5786 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.211931    5786 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212016    5786 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212023    5786 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:12:24.221642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212029    5786 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212034    5786 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212045    5786 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212050    5786 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212057    5786 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212096    5786 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212105    5786 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212112    5786 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212118    5786 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212124    5786 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212141    5786 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212146    5786 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212172    5786 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212194    5786 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212199    5786 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212237    5786 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212245    5786 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212257    5786 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212262    5786 flags.go:64] FLAG: --v="2"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212270    5786 flags.go:64] FLAG: --version="false"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212277    5786 flags.go:64] FLAG: --vmodule=""
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212284    5786 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.212455    5786 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213132    5786 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213158    5786 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:12:24.222851 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213166    5786 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213170    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213174    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213179    5786 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213183    5786 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213188    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213194    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213198    5786 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213231    5786 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213242    5786 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213248    5786 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213260    5786 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213264    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213270    5786 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213275    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213280    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213285    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213314    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213322    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213327    5786 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:12:24.223751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213331    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213335    5786 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213346    5786 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213351    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213355    5786 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213361    5786 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213366    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213391    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213398    5786 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213402    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213407    5786 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213412    5786 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213416    5786 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213421    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213430    5786 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213434    5786 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213438    5786 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213442    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213447    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:12:24.224417 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213481    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213491    5786 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213495    5786 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213500    5786 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213504    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213508    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213512    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213517    5786 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213526    5786 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213531    5786 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213535    5786 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213561    5786 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213566    5786 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213570    5786 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213575    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213580    5786 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213585    5786 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213590    5786 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213602    5786 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:12:24.224976 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213607    5786 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213614    5786 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213649    5786 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213654    5786 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213659    5786 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213664    5786 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213669    5786 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213674    5786 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213678    5786 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213683    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213696    5786 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213716    5786 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213722    5786 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213727    5786 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213749    5786 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213753    5786 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213780    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213787    5786 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213792    5786 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213797    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:12:24.225621 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213803    5786 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213808    5786 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213815    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213820    5786 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213825    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213830    5786 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213864    5786 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213877    5786 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213883    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213888    5786 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213892    5786 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213898    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213909    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213913    5786 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213948    5786 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213956    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213964    5786 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213971    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213976    5786 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213987    5786 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:12:24.226449 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213992    5786 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.213996    5786 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214024    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214031    5786 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214035    5786 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214040    5786 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.214144    5786 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214508    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214530    5786 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214537    5786 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214574    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214583    5786 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214588    5786 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214593    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214598    5786 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214605    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:12:24.227362 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214610    5786 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214614    5786 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214618    5786 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214654    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214666    5786 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214672    5786 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214678    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214683    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214688    5786 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214692    5786 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214697    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214726    5786 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214741    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214747    5786 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214751    5786 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214762    5786 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214767    5786 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214772    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214779    5786 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:12:24.228011 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214808    5786 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214814    5786 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214819    5786 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214824    5786 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214828    5786 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214834    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214838    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214843    5786 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214848    5786 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214857    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214861    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214888    5786 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214899    5786 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214904    5786 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214908    5786 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214912    5786 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214916    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214921    5786 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214926    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214930    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:12:24.228752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214935    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214968    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214975    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214980    5786 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214985    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214990    5786 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.214996    5786 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215000    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215005    5786 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215033    5786 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215040    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215045    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215051    5786 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215060    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215069    5786 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215074    5786 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215078    5786 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215083    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215089    5786 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:12:24.229540 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215120    5786 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215125    5786 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215130    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215134    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215139    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215145    5786 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215149    5786 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215155    5786 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215164    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215169    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215194    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215229    5786 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215236    5786 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215241    5786 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215252    5786 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215274    5786 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215279    5786 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215285    5786 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215290    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:12:24.230280 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215295    5786 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215305    5786 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215309    5786 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215314    5786 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215343    5786 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215350    5786 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215356    5786 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215361    5786 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215366    5786 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215371    5786 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215375    5786 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215390    5786 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215428    5786 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215434    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215438    5786 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215443    5786 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215447    5786 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215452    5786 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215458    5786 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.215462    5786 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:12:24.231030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.225150    5786 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.225175    5786 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.225235    5786 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.225245    5786 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225515    5786 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225527    5786 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225532    5786 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225536    5786 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225562    5786 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225568    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225573    5786 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225577    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225581    5786 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225586    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225590    5786 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225596    5786 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225606    5786 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225611    5786 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:12:24.231774 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225616    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225633    5786 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225638    5786 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225642    5786 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225647    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225651    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225655    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225661    5786 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225666    5786 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225670    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225674    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225683    5786 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225688    5786 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225692    5786 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225698    5786 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225702    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225707    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225714    5786 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225721    5786 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:12:24.232223 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225727    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225732    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225737    5786 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225741    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225750    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225755    5786 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225760    5786 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225764    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225769    5786 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225773    5786 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225778    5786 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225782    5786 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225787    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225791    5786 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225795    5786 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225800    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225809    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225814    5786 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225818    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225823    5786 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:12:24.232688 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225827    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225836    5786 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225840    5786 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225845    5786 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225850    5786 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225855    5786 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225859    5786 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225864    5786 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225869    5786 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225878    5786 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225882    5786 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225886    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225890    5786 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225896    5786 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225900    5786 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225904    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225908    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225913    5786 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225917    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225921    5786 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:12:24.233340 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225926    5786 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225930    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225939    5786 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225943    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225948    5786 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225952    5786 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225957    5786 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225961    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225965    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225969    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225973    5786 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225977    5786 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225982    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.225986    5786 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226004    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226011    5786 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226018    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226024    5786 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226028    5786 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:12:24.233835 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226033    5786 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226037    5786 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226041    5786 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226045    5786 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226050    5786 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226054    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226058    5786 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226062    5786 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226071    5786 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226075    5786 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226080    5786 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226084    5786 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226088    5786 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226092    5786 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.226151    5786 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226728    5786 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:12:24.234308 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226739    5786 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226746    5786 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226751    5786 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226756    5786 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226761    5786 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226772    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226776    5786 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226780    5786 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226784    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226788    5786 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226792    5786 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226796    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226807    5786 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226812    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226817    5786 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226822    5786 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226826    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226835    5786 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226839    5786 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:12:24.234687 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226843    5786 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226847    5786 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226851    5786 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226855    5786 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226860    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226864    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226868    5786 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226872    5786 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226876    5786 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226881    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226885    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226894    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226899    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226904    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226908    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226912    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226917    5786 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226921    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226925    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:12:24.235143 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226930    5786 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226934    5786 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226938    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226942    5786 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226946    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226955    5786 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226959    5786 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226970    5786 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226975    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226979    5786 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226984    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226988    5786 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226993    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.226997    5786 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227002    5786 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227006    5786 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227011    5786 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227015    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227025    5786 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227029    5786 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:12:24.235717 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227034    5786 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227039    5786 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227043    5786 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227048    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227054    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227058    5786 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227063    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227066    5786 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227071    5786 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227076    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227084    5786 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227088    5786 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227093    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227097    5786 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227102    5786 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227105    5786 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227109    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227113    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227116    5786 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:12:24.236431 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227121    5786 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227133    5786 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227140    5786 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227161    5786 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227168    5786 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227173    5786 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227178    5786 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227182    5786 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227186    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227193    5786 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227198    5786 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227228    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227234    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227239    5786 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227244    5786 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227248    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227258    5786 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227262    5786 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227266    5786 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:12:24.237039 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227270    5786 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227275    5786 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227279    5786 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227283    5786 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227287    5786 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227292    5786 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227296    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227300    5786 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227304    5786 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.227369    5786 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227520    5786 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227529    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227534    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227539    5786 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227547    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227559    5786 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:12:24.237863 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227564    5786 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227569    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227574    5786 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227579    5786 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227589    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227595    5786 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227600    5786 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227605    5786 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227610    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227614    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227618    5786 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227622    5786 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227627    5786 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227633    5786 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227640    5786 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227647    5786 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227658    5786 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227664    5786 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227669    5786 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:12:24.238506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227674    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227680    5786 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227686    5786 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227691    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227696    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227700    5786 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227705    5786 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227710    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227715    5786 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227724    5786 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227729    5786 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227735    5786 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227739    5786 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227750    5786 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227755    5786 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227760    5786 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227764    5786 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227791    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227811    5786 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:12:24.239242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227817    5786 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227822    5786 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227828    5786 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227837    5786 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227842    5786 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227846    5786 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227850    5786 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227854    5786 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227858    5786 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227862    5786 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227867    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227873    5786 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227877    5786 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227881    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227886    5786 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227896    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227901    5786 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227906    5786 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227911    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227915    5786 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:12:24.240002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227920    5786 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227924    5786 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227929    5786 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227933    5786 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227946    5786 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227951    5786 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227955    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227966    5786 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227975    5786 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227979    5786 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227983    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227988    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227993    5786 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.227997    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228001    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228005    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228009    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228013    5786 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228018    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:12:24.240918 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228026    5786 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228036    5786 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228042    5786 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228046    5786 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228053    5786 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228058    5786 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228062    5786 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228066    5786 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228070    5786 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228075    5786 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228079    5786 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228083    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228087    5786 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228091    5786 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228100    5786 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228104    5786 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228109    5786 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228115    5786 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228119    5786 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228124    5786 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:12:24.241426 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228128    5786 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228140    5786 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:24.228145    5786 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.228320    5786 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.232647    5786 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.232747    5786 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.233024    5786 server.go:1014] "Starting client certificate rotation"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.233200    5786 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.233291    5786 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:06:07 +0000 UTC" deadline="2028-04-16 09:21:51.254698525 +0000 UTC"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.233353    5786 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12804h9m27.021349459s"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.233615    5786 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.235826    5786 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:12:24.241929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.240771    5786 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:12:24.244165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.244149    5786 log.go:25] "Validated CRI v1 image API"
Oct 30 21:12:24.245540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.245525    5786 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:12:24.249407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.249371    5786 fs.go:135] Filesystem UUIDs: map[07be4522-0d4c-4744-8a0f-79c744ca9085:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a44f5419-2dab-4964-8cc8-c35488adeee4:/dev/nvme0n1p4]
Oct 30 21:12:24.253489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.249407    5786 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0983ffbaeae93fc95074f36179d59011469de179bc13013c28324019604ceaa8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0983ffbaeae93fc95074f36179d59011469de179bc13013c28324019604ceaa8/userdata/shm major:0 minor:306 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/101b3b78fdd561d363adeae439f500559787c01a606ed10d82262eb0e5bb4c88/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/101b3b78fdd561d363adeae439f500559787c01a606ed10d82262eb0e5bb4c88/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1900df2fe84054e14327a744aedb763f60f61f4af17f5c749f6396379cc8c797/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1900df2fe84054e14327a744aedb763f60f61f4af17f5c749f6396379cc8c797/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2dea41af0a258a3b1a12868035a38ac48c13b4da0a66207a22e2e2058183566f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2dea41af0a258a3b1a12868035a38ac48c13b4da0a66207a22e2e2058183566f/userdata/shm major:0 minor:352 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/340a3c8fdb92d2bbf407cf693effa8ac046ac56ebcde9ce08714e4cbf2d6942a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/340a3c8fdb92d2bbf407cf693effa8ac046ac56ebcde9ce08714e4cbf2d6942a/userdata/shm major:0 minor:364 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3b894f85b2fe38d45531b3f85ba1527ec36cd0d53445805e280b22c3026e304d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3b894f85b2fe38d45531b3f85ba1527ec36cd0d53445805e280b22c3026e304d/userdata/shm major:0 minor:426 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4dcb576a5d7db4d871f30cff4869ae4b04f4b2a68da30d8557f3c8eb09076577/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4dcb576a5d7db4d871f30cff4869ae4b04f4b2a68da30d8557f3c8eb09076577/userdata/shm major:0 minor:413 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5a5a34150e3e7a1212eb3d96c6573ba1e59c5cfdc8f697c2cd859e39023b6340/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5a5a34150e3e7a1212eb3d96c6573ba1e59c5cfdc8f697c2cd859e39023b6340/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/71604d9195264339523ac2dbfb9f5b3ea6ab2630c51ce057100d9e7e7e0e6a2e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/71604d9195264339523ac2dbfb9f5b3ea6ab2630c51ce057100d9e7e7e0e6a2e/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/731e40fc0bbacca627cc99edeba47227c1817c2834bc4ac6d75ed734d933f4d0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/731e40fc0bbacca627cc99edeba47227c1817c2834bc4ac6d75ed734d933f4d0/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/77a7dbfc9efa8f6193edf5e13551c0fb5315d9c0570130c9f44cf06d07323780/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/77a7dbfc9efa8f6193edf5e13551c0fb5315d9c0570130c9f44cf06d07323780/userdata/shm major:0 minor:50 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8606df236e3ab41a1bcfe7bc2f8ed734cb041b463d81cf938f901d7ed960f316/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8606df236e3ab41a1bcfe7bc2f8ed734cb041b463d81cf938f901d7ed960f316/userdata/shm major:0 minor:256 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/880a3aa2d1320f60b4091c342d93ce7938629e06079302dd3592d8a58d7029eb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/880a3aa2d1320f60b4091c342d93ce7938629e06079302dd3592d8a58d7029eb/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b176117b104c6ba75d453da4c5f021bd3fca01295c2ee5af4a5734afce77905d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b176117b104c6ba75d453da4c5f021bd3fca01295c2ee5af4a5734afce77905d/userdata/shm major:0 minor:287 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b6831c3f2c9e20d74d9c8847890d323aa22d875cc5e18d2da3153310ead2c24a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b6831c3f2c9e20d74d9c8847890d323aa22d875cc5e18d2da3153310ead2c24a/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bb27ebef79a610f4e6e4143ebc4253f4a644cfd67a8799f63d18b90f9bbab747/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bb27ebef79a610f4e6e4143ebc4253f4a644cfd67a8799f63d18b90f9bbab747/userdata/shm major:0 minor:312 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c093e55397c67fa6a5936baf5661fb91db7544b1766d7b62798baac63bfecf32/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c093e55397c67fa6a5936baf5661fb91db7544b1766d7b62798baac63bfecf32/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cb1e06307a77553a8b67e40b8f6b32f03e00357c369580f4e9f635e90498f4ff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cb1e06307a77553a8b67e40b8f6b32f03e00357c369580f4e9f635e90498f4ff/userdata/shm major:0 minor:310 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc186bbec82ab8f459d5782feb8f03feff5eb8f492b065a5b9163e9f554ff811/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc186bbec82ab8f459d5782feb8f03feff5eb8f492b065a5b9163e9f554ff811/userdata/shm major:0 minor:269 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ced26a1d46e50a01617506a794137dc1eef61adb965bbc11eee56429b097313a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ced26a1d46e50a01617506a794137dc1eef61adb965bbc11eee56429b097313a/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d226dc6231c66ddbe1c414b394439e2e2ffa5227b4ed96858749c92d531b0c5a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d226dc6231c66ddbe1c414b394439e2e2ffa5227b4ed96858749c92d531b0c5a/userdata/shm major:0 minor:407 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d36cd8c38b4843bcb1a14c2991627a555637dac86ed828f75db32dc4cc8e93b7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d36cd8c38b4843bcb1a14c2991627a555637dac86ed828f75db32dc4cc8e93b7/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d3fa2a371f2d6036f543276331a4fb158e2aee350eca63a40a89e18a0747346b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d3fa2a371f2d6036f543276331a4fb158e2aee350eca63a40a89e18a0747346b/userdata/shm major:0 minor:251 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d60eaa159a472139d1e983915b0ee2aaaa5faff5a84b30404dbc2d8457c12066/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d60eaa159a472139d1e983915b0ee2aaaa5faff5a84b30404dbc2d8457c12066/userdata/shm major:0 minor:372 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e12a94578f03e5c94a708a1f65d538b8641c5615f95421e15c5406b1b7a952cf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e12a94578f03e5c94a708a1f65d538b8641c5615f95421e15c5406b1b7a952cf/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e87150dd997f161a0deb79703155a9b007c0cb011f1b3b7933f07df4f6168b10/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e87150dd997f161a0deb79703155a9b007c0cb011f1b3b7933f07df4f6168b10/userdata/shm major:0 minor:291 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f62229a1a7ae4d01bc19ee2116c7b93d5aecf12c8d00ac2aa80039d1e662dad2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f62229a1a7ae4d01bc19ee2116c7b93d5aecf12c8d00ac2aa80039d1e662dad2/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb134c75282a241532a64707fd797de266507d282984a9c0d671a6aa71cf1134/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb134c75282a241532a64707fd797de266507d282984a9c0d671a6aa71cf1134/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0492a73c-e41b-46c7-ad57-1f8220dfcf3a/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/0492a73c-e41b-46c7-ad57-1f8220dfcf3a/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06846bf4-72b9-4348-b4ae-5108221dbd0b/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/06846bf4-72b9-4348-b4ae-5108221dbd0b/volumes/kubernetes.io~secret/agent-certs major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~projected/kube-api-access-swj72:{mountpoint:/var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~projected/kube-api-access-swj72 major:0 minor:309 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:305 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1daa2700-153a-4fed-8434-669dd9b021bb/volumes/kubernetes.io~projected/kube-api-access-68rr5:{mountpoint:/var/lib/kubelet/pods/1daa2700-153a-4fed-8434-669dd9b021bb/volumes/kubernetes.io~projected/kube-api-access-68rr5 major:0 minor:255 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e4cd6ee-1f05-473a-b32c-d9a53a790541/volumes/kubernetes.io~projected/kube-api-access-px5f9:{mountpoint:/var/lib/kubelet/pods/1e4cd6ee-1f05-473a-b32c-d9a53a790541/volumes/kubernetes.io~projected/kube-api-access-px5f9 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~projected/kube-api-access-jx5b5:{mountpoint:/var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~projected/kube-api-access-jx5b5 major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2238e566-a39a-47ff-8f14-a27a404bc255/volumes/kubernetes.io~projected/kube-api-access-c9pvw:{mountpoint:/var/lib/kubelet/pods/2238e566-a39a-47ff-8f14-a27a404bc255/volumes/kubernetes.io~projected/kube-api-access-c9pvw major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2238e566-a39a-47ff-8f14-a27a404bc255/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/2238e566-a39a-47ff-8f14-a27a404bc255/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~projected/kube-api-access-pl9k5:{mountpoint:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~projected/kube-api-access-pl9k5 major:0 minor:425 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:421 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:424 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:422 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:423 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a5786b3-e466-4fc8-99c6-bc2876267401/volumes/kubernetes.io~projected/kube-api-access-gsn2n:{mountpoint:/var/lib/kubelet/pods/4a5786b3-e466-4fc8-99c6-bc2876267401/volumes/kubernetes.io~projected/kube-api-access-gsn2n major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~projected/kube-api-access-kjbpj:{mountpoint:/var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~projected/kube-api-access-kjbpj major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:267 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b61fec4-1b8d-449c-b251-b48c3306d351/volumes/kubernetes.io~projected/kube-api-access-9mbqj:{mountpoint:/var/lib/kubelet/pods/6b61fec4-1b8d-449c-b251-b48c3306d351/volumes/kubernetes.io~projected/kube-api-access-9mbqj major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b61fec4-1b8d-449c-b251-b48c3306d351/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/6b61fec4-1b8d-449c-b251-b48c3306d351/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7535ee53-f698-47f2-83fa-e0694379654a/volumes/kubernetes.io~secret/tls-certificates:{mountpoint:/var/lib/kubelet/pods/7535ee53-f698-47f2-83fa-e0694379654a/volumes/kubernetes.io~secret/tls-certificates major:0 minor:247 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8224a234-a815-4d78-9960-1c226c1224e4/volumes/kubernetes.io~projected/kube-api-access-84ntf:{mountpoint:/var/lib/kubelet/pods/8224a234-a815-4d78-9960-1c226c1224e4/volumes/kubernetes.io~projected/kube-api-access-84ntf major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~projected/kube-api-access-wxd9w:{mountpoint:/var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~projected/kube-api-access-wxd9w major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~projected/kube-api-access-vzngr:{mountpoint:/var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~projected/kube-api-access-vzngr major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volumes/kubernetes.io~projected/kube-api-access-dmf5z:{mountpoint:/var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volumes/kubernetes.io~projected/kube-api-access-dmf5z major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99ddf84f-2164-4309-a9bb-30a816909083/volumes/kubernetes.io~projected/kube-api-access-nwkmt:{mountpoint:/var/lib/kubelet/pods/99ddf84f-2164-4309-a9bb-30a816909083/volumes/kubernetes.io~projected/kube-api-access-nwkmt major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~projected/kube-api-access-n6kcg:{mountpoint:/var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~projected/kube-api-access-n6kcg major:0 minor:308 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b8b7c30e-9edc-43f6-ba65-279d40a3267e/volumes/kubernetes.io~projected/kube-api-access-9w9zx:{mountpoint:/var/lib/kubelet/pods/b8b7c30e-9edc-43f6-ba65-279d40a3267e/volumes/kubernetes.io~projected/kube-api-access-9w9zx major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b8b7c30e-9edc-43f6-ba65-279d40a3267e/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/b8b7c30e-9edc-43f6-ba65-279d40a3267e/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/alertmanager/9 major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/config-reloader/6 major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~empty-dir/config-out major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~projected/kube-api-access-rgck7:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~projected/kube-api-access-rgck7 major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~projected/tls-assets major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/config-volume major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:346 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:349 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/web-config major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1a59846-fdda-4740-ada7-13de6a939e93/volumes/kubernetes.io~projected/kube-api-access-qt65s:{mountpoint:/var/lib/kubelet/pods/d1a59846-fdda-4740-ada7-13de6a939e93/volumes/kubernetes.io~projected/kube-api-access-qt65s major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e23751d7-2d21-4805-8a90-fd6276de3545/volumes/kubernetes.io~projected/kube-api-access-z4b99:{mountpoint:/var/lib/kubelet/pods/e23751d7-2d21-4805-8a90-fd6276de3545/volumes/kubernetes.io~projected/kube-api-access-z4b99 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~projected/kube-api-access-sc2mr:{mountpoint:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~projected/kube-api-access-sc2mr major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:354 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:359 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:358 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f804883e-c5eb-46e0-a6d2-64db96efa860/volumes/kubernetes.io~projected/kube-api-access-zh9l9:{mountpoint:/var/lib/kubelet/pods/f804883e-c5eb-46e0-a6d2-64db96efa860/volumes/kubernetes.io~projected/kube-api-access-zh9l9 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~projected/kube-api-access-v8vsb:{mountpoint:/var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~projected/kube-api-access-v8vsb major:0 minor:412 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:411 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:410 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:409 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe2359dc-5156-4bd5-8e7f-1bce48d948e6/volumes/kubernetes.io~projected/kube-api-access-nssz5:{mountpoint:/var/lib/kubelet/pods/fe2359dc-5156-4bd5-8e7f-1bce48d948e6/volumes/kubernetes.io~projected/kube-api-access-nssz5 major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe2359dc-5156-4bd5-8e7f-1bce48d948e6/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/fe2359dc-5156-4bd5-8e7f-1bce48d948e6/volumes/kubernetes.io~secret/metrics-certs major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe7b8332-a94f-40a0-b4b5-5880f10b571a/volumes/kubernetes.io~projected/kube-api-access-hlk7x:{mountpoint:/var/lib/kubelet/pods/fe7b8332-a94f-40a0-b4b5-5880f10b571a/volumes/kubernetes.io~projected/kube-api-access-hlk7x major:0 minor:406 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe7b8332-a94f-40a0-b4b5-5880f10b571a/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/fe7b8332-a94f-40a0-b4b5-5880f10b571a/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:402 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/352a595e362b313bd34b6f152753756e8ac9d872c758b9e9175a07fc871d8b2d/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/da462750fc9295cf0d5acbefd3b3682e66f070f1733d82c7dd7b1086eebdd4bf/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/e13872c1b4e1b8dd66fb71b71c960fce7182b483f460d6ca4f6b73eed892a9e4/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/cfbfd51fb42769545b8d9476f85ef0e595a5296392a16472c5fc6a883d094f60/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/227e2586aeff7feb2640a8607e45af7700decae35b9760603ca99d7f54ee1769/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/529288b0b4bf23ddf3dd470c81d3bcbcef865d3861eb830194305055388e5553/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/86f79da47667bb3358c3bb95cb9c9b7ed8fe50b07272767795dc2c94ed8854d6/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/b83c1cb2cca69fb54b6108e7f341b9d6ff5d22ae8d402b0007b7036d30419fe4/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/74cbb0dd6c120312d6d1464d87d7ecd082743051e0bea4a1d323928b268771ee/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/1b1c6571f587161e1d98ecaceabd69a034a34b6b572f21bad5c279d70502cdfd/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/5202c6d0cbe83900318a8c2c27aeb3417b06864ae719c63ea96445d7454f22a4/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/e3788c33dbaca6c6e01d1f0fda42b447f9aedde15e29020fcfbdf9f6dfa5a30c/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/aeec2271c1f9565fe9d40906658e032dbe95c5e322132b43aab7f94a48d5b06c/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/b4d48ea4bb876fbfa2ba269f00b9082959877eeda4abdef289c6255936522e95/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/ab1f3185849a4975b8dc3aad0e5fe113abecae4ad89e78332b55d4ed77c2c37c/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/77922f98cd2cdaa64ea940d7dd31ea71d6b78440c4402f91356f6eb2758081ec/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/a5fba46a9aead564ffc7b65dee8e02111dec9edb5fb6945e5ddc25a8391e230e/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/f247bae271db00e9a333e4e77a35a92c1ccdc19343e6f97b47dee3b61300b912/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/519245bca8f202ac874dbb2ae219ea32abbe6faf51f56de2a3597d6cb071670f/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/76a2ab240271c6e7faff151a34b921724c53be096430c98e8bed9254e38565de/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/cd581205b8c59505b4b5802d026dc4d91dca8ac580972fa6dbe45c7e3a2a64fe/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/6093b06b51861b19e0497fcc42fe8e19b99fec9f976e9c54ec58fe6e742ea3ed/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/dcbed018da255ec0d918093da1e1644fef4286cb5f6a43f9530f3175022d1330/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/39c083e6c3159856c20e3cf62a4cf71f3b4db4234a94285fbf29816ed21bf259/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/ca472aecd896e614001fa9197457424dac57e37ff5728ce43925672c5255f2e6/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/bf35f8428931293b3c24e2af60f80b9cbd1b897cad3ad8e762117505eddae749/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/16b715e0121ff8a4b0de6c49c681a9799c29dacb2852c5c231a661e457ca3e46/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/6b3b9806e2b9b08d81cc95a144829c1953e95dc0e30f26b9b77809e63f531d0c/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/ea898d27701f6af661eda09b0e53574ee230e233071e4047716ea594da330d54/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/e426501b452a8645496ef6d125f9a8a1c1a01f93841ff0b32d3aa5f60409b7e0/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/059252c7338e27cb0eff26dada536fe08b4531d78d504bb08fc5a8696d660bf7/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/c9e125cb6a61aa2b68b8e9b554a6d18d1df8e7c0c6135e5053595782b29ee51f/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/afb8f7ae2f8c0d7fa244b1d97e1ba5a8df039a39cc714cb0a24376aae9a4fdfa/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/7c00fec8ec8a21d73787669873b4216923bc05085eba77dc59fb40bf4fd92ff1/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/d9c4e2247dd52ab2217b491fbb9f23143c2da5eb59a7a1fe118272bce02bb9fd/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/6cd492604e99a007bccb5121aaacefaf1e99b2ce9acd9601c714e6d6a337477e/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/4a9886a4bfc4b21f7533e593e9348fa627caddda74eba756ad5a99be4222d1a7/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-295:{mountpoint:/var/lib/containers/storage/overlay/d75ac2bf788d6d6d37b3bde4a1466102e3ce7ce6f9a81ac22c90fce65920d26c/merged major:0 minor:295 fsType:overlay blockSize:0} overlay_0-297:{mountpoint:/var/lib/containers/storage/overlay/1315f140bfae0d2c052f5ce94ecadbca4be71c6adf80a59823db44c7286cb9ff/merged major:0 minor:297 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/041ea9529d5ca630e84eb148a416ba9b4aa42b24abf54236ca665c5f8825615d/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/4984145d444d9026ba144db168fcf85e6f2e5c8648dc9b534cc1014f7cd7e2c1/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/ae53cbee2de51cd2c0296a8d6b9a29b4ccd13f30c506026cfe9d6a82831df4c2/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/98d2b4915700f532205268e312f4714de2bfcdf0e54e866e58d2b2d669ab1807/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/fc20e9b47d02313af5dfda095c895431170be9e9c870e4a0c5784e64188b0482/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/e2afd4cdb5d6d1948846512655d528995e034f61e00696981f28b4b34fc57e5c/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-333:{mountpoint:/var/lib/containers/storage/overlay/d6860ce87d7f23240250c3de1faafd627faa8cfb8eb8cb02c7344e359cae3dfc/merged major:0 minor:333 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/6ccbdca0e0a60ef7da7f457cd3d62659c7cf974d51b17c5a5a35dbaa27eea2c9/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/d890cf2ac3c26dcaa4e1dbb1f70c9ff9e5634ad5efcf71d5b587103bbe1ab0b5/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/9c7bf5362a458707c10ab8d627ca65f681a8d94b8f58ac071e31361f7aadef9f/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/8bdebd263e115e0b0f0a94818be16e6aa2a2c7a07e0f3d967aa8f801fa73ad41/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-382:{mountpoint:/var/lib/containers/storage/overlay/1eb25b8fcb0f6e4f6cccaf9859f5748c1ca339eed9e9a7bea8943657f55a339e/merged major:0 minor:382 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/b05dac0fd69d2e4bc0e921e8b72c0726358f56df70be5f2045b9a1e38252acc4/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-389:{mountpoint:/var/lib/containers/storage/overlay/8ff6044bfd4a9a08f8e1ec6a4f7bc1cf2692c5e9c973328af76087eafe7af0bb/merged major:0 minor:389 fsType:overlay blockSize:0} overlay_0-391:{mountpoint:/var/lib/containers/storage/overlay/86b84588d4fed357ded473d27e735f95b9f2503c3ee9b5bd6ae36e82ad647815/merged major:0 minor:391 fsType:overlay blockSize:0} overlay_0-397:{mountpoint:/var/lib/containers/storage/overlay/11a9ea8d678977229b1f9bf40bd659b353fb3ead7bfa9789517d0f7450969bec/merged major:0 minor:397 fsType:overlay blockSize:0} overlay_0-415:{mountpoint:/var/lib/containers/storage/overlay/eeabfbe23c81d234d97a60397e5c355aee970471ae25089f0fa0db9192f6564e/merged major:0 minor:415 fsType:overlay blockSize:0} overlay_0-417:{mountpoint:/var/lib/containers/storage/overlay/a835d1c5dc6f0e7348e34146d582ef513a63c6c00ca102817c596d8913a32163/merged major:0 minor:417 fsType:overlay blockSize:0} overlay_0-419:{mountpoint:/var/lib/containers/storage/overlay/716a9142e1110a189f87dc0f2e7092ceae9f500e15c9515982e27e701074259f/merged major:0 minor:419 fsType:overlay blockSize:0} overlay_0-432:{mountpoint:/var/lib/containers/storage/overlay/ddb861d79e2cad5df201519590563e99b460a20530d7664e98134dc9639d4da3/merged major:0 minor:432 fsType:overlay blockSize:0} overlay_0-434:{mountpoint:/var/lib/containers/storage/overlay/268c42aec13468496d99c8025afddaf3647e45dd4ed9a46898636a02627b531a/merged major:0 minor:434 fsType:overlay blockSize:0} overlay_0-436:{mountpoint:/var/lib/containers/storage/overlay/3dc96d0abd15f8d0f1a7befcd3e3a78dd3d5ea92687c9d96a3b38e039424c77a/merged major:0 minor:436 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/e50e3625d9ccbf9e798166a92edbfae322e7d96074a026d67d7cb9008fb310ce/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-447:{mountpoint:/var/lib/containers/storage/overlay/b8b1dec676f9fb79cc73706389604d844bcfb642b6bab7054471d90eda0e5292/merged major:0 minor:447 fsType:overlay blockSize:0} overlay_0-452:{mountpoint:/var/lib/containers/storage/overlay/da3348d588a4b07b541e0049e08a0792cf873caefa1197d703c3d9c014d2631b/merged major:0 minor:452 fsType:overlay blockSize:0} overlay_0-454:{mountpoint:/var/lib/containers/storage/overlay/c748a691a5e3893c51656f56b700ed4281eccac8ead772991c6d90efaf964ce0/merged major:0 minor:454 fsType:overlay blockSize:0} overlay_0-456:{mountpoint:/var/lib/containers/storage/overlay/e8eb43721a33362fe1561a9cc6bc5bcf3f5bc80215c2994ddbc80a45525ae33d/merged major:0 minor:456 fsType:overlay blockSize:0} overlay_0-466:{mountpoint:/var/lib/containers/storage/overlay/6014669dadee2c8e5b004a9f5efa0d8e115aa2ba51daeb03c07f9b07b73998ac/merged major:0 minor:466 fsType:overlay blockSize:0} overlay_0-472:{mountpoint:/var/lib/containers/storage/overlay/830a2e21b191634beb5bbc9732a55361807fad051781d6ae323f551b24a63c04/merged major:0 minor:472 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/13e91710ef06c44c094eb4af9c996fd7fa08378c623d0cfe9b6d1874d5384957/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/21adca6f2e83bc68e91e7a155ae63910de8727ca75df34f232a02fb84453589e/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/fe8a9ab2cce485ab8d0dc8a87a67e158b015b276983cc857a03b4122a1e5bf31/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/9dd195eed1c0c4ea082f89c8ba7d945e589d474f65dcad1672b7ded1c71ef095/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/883e4f413b72f06b506758d49c0531f12f5d0e2ef0a494dfd4c8581226c51e76/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/16f426fb87370dea95400717463d0e13be17fa131b1449186656b64fb9e944d7/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/a1a6a81668a71e3142c3b6c8066301cc6870a871a0a4013cc5cfb99575e8da6d/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/11b1c788b1a64f83b5d11ec5ec20c9f57a2b29560af80999330276228cfe66d8/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/5394948e13422c477c9d197fa9758dca33ee98267df0dd3f048958124bd8ccc9/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/bce0a2acb5a9dd9c252948a94085cdae911ddc08bc8817332f5a6d6c29de7e2d/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 30 21:12:24.262454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.261777    5786 manager.go:217] Machine: {Timestamp:2026-10-30 21:12:24.260953634 +0000 UTC m=+0.132072749 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102220 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2627451245dedc35810aaceb3d0358 SystemUUID:ec262745-1245-dedc-3581-0aaceb3d0358 BootID:2c2d3cb3-0532-4cb2-b20f-679ae91aaa6d Filesystems:[{Device:/var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:262 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:345 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:348 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~projected/kube-api-access-vzngr DeviceMajor:0 DeviceMinor:301 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-432 DeviceMajor:0 DeviceMinor:432 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-456 DeviceMajor:0 DeviceMinor:456 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-466 DeviceMajor:0 DeviceMinor:466 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:347 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-297 DeviceMajor:0 DeviceMinor:297 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0983ffbaeae93fc95074f36179d59011469de179bc13013c28324019604ceaa8/userdata/shm DeviceMajor:0 DeviceMinor:306 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-389 DeviceMajor:0 DeviceMinor:389 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~projected/kube-api-access-kjbpj DeviceMajor:0 DeviceMinor:268 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~projected/kube-api-access-sc2mr DeviceMajor:0 DeviceMinor:360 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d226dc6231c66ddbe1c414b394439e2e2ffa5227b4ed96858749c92d531b0c5a/userdata/shm DeviceMajor:0 DeviceMinor:407 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~projected/kube-api-access-v8vsb DeviceMajor:0 DeviceMinor:412 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0492a73c-e41b-46c7-ad57-1f8220dfcf3a/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:281 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cb1e06307a77553a8b67e40b8f6b32f03e00357c369580f4e9f635e90498f4ff/userdata/shm DeviceMajor:0 DeviceMinor:310 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-419 DeviceMajor:0 DeviceMinor:419 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-436 DeviceMajor:0 DeviceMinor:436 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b6831c3f2c9e20d74d9c8847890d323aa22d875cc5e18d2da3153310ead2c24a/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~projected/kube-api-access-n6kcg DeviceMajor:0 DeviceMinor:308 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2dea41af0a258a3b1a12868035a38ac48c13b4da0a66207a22e2e2058183566f/userdata/shm DeviceMajor:0 DeviceMinor:352 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-397 DeviceMajor:0 DeviceMinor:397 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f804883e-c5eb-46e0-a6d2-64db96efa860/volumes/kubernetes.io~projected/kube-api-access-zh9l9 DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:300 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:422 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1e4cd6ee-1f05-473a-b32c-d9a53a790541/volumes/kubernetes.io~projected/kube-api-access-px5f9 DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/731e40fc0bbacca627cc99edeba47227c1817c2834bc4ac6d75ed734d933f4d0/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f62229a1a7ae4d01bc19ee2116c7b93d5aecf12c8d00ac2aa80039d1e662dad2/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1900df2fe84054e14327a744aedb763f60f61f4af17f5c749f6396379cc8c797/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b8b7c30e-9edc-43f6-ba65-279d40a3267e/volumes/kubernetes.io~projected/kube-api-access-9w9zx DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55bed0f4-2cfe-4624-90b8-2f56b728d578/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:267 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-333 DeviceMajor:0 DeviceMinor:333 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-391 DeviceMajor:0 DeviceMinor:391 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc186bbec82ab8f459d5782feb8f03feff5eb8f492b065a5b9163e9f554ff811/userdata/shm DeviceMajor:0 DeviceMinor:269 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:423 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/880a3aa2d1320f60b4091c342d93ce7938629e06079302dd3592d8a58d7029eb/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1daa2700-153a-4fed-8434-669dd9b021bb/volumes/kubernetes.io~projected/kube-api-access-68rr5 DeviceMajor:0 DeviceMinor:255 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe2359dc-5156-4bd5-8e7f-1bce48d948e6/volumes/kubernetes.io~projected/kube-api-access-nssz5 DeviceMajor:0 DeviceMinor:53 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d36cd8c38b4843bcb1a14c2991627a555637dac86ed828f75db32dc4cc8e93b7/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6b61fec4-1b8d-449c-b251-b48c3306d351/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6b61fec4-1b8d-449c-b251-b48c3306d351/volumes/kubernetes.io~projected/kube-api-access-9mbqj DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:303 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~projected/kube-api-access-wxd9w DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/77a7dbfc9efa8f6193edf5e13551c0fb5315d9c0570130c9f44cf06d07323780/userdata/shm DeviceMajor:0 DeviceMinor:50 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2238e566-a39a-47ff-8f14-a27a404bc255/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:410 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-415 DeviceMajor:0 DeviceMinor:415 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-472 DeviceMajor:0 DeviceMinor:472 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-295 DeviceMajor:0 DeviceMinor:295 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:304 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-417 DeviceMajor:0 DeviceMinor:417 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:347 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb134c75282a241532a64707fd797de266507d282984a9c0d671a6aa71cf1134/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:305 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:362 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volumes/kubernetes.io~projected/kube-api-access-dmf5z DeviceMajor:0 DeviceMinor:54 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d3fa2a371f2d6036f543276331a4fb158e2aee350eca63a40a89e18a0747346b/userdata/shm DeviceMajor:0 DeviceMinor:251 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8606df236e3ab41a1bcfe7bc2f8ed734cb041b463d81cf938f901d7ed960f316/userdata/shm DeviceMajor:0 DeviceMinor:256 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5a5a34150e3e7a1212eb3d96c6573ba1e59c5cfdc8f697c2cd859e39023b6340/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e87150dd997f161a0deb79703155a9b007c0cb011f1b3b7933f07df4f6168b10/userdata/shm DeviceMajor:0 DeviceMinor:291 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:339 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:350 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b8b7c30e-9edc-43f6-ba65-279d40a3267e/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/92966fe2-fe0f-494f-9bc8-87000dd086c9/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:299 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:348 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:357 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe7b8332-a94f-40a0-b4b5-5880f10b571a/volumes/kubernetes.io~projected/kube-api-access-hlk7x DeviceMajor:0 DeviceMinor:406 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c093e55397c67fa6a5936baf5661fb91db7544b1766d7b62798baac63bfecf32/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~projected/kube-api-access-rgck7 DeviceMajor:0 DeviceMinor:351 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:359 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3b894f85b2fe38d45531b3f85ba1527ec36cd0d53445805e280b22c3026e304d/userdata/shm DeviceMajor:0 DeviceMinor:426 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~projected/kube-api-access-jx5b5 DeviceMajor:0 DeviceMinor:363 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/340a3c8fdb92d2bbf407cf693effa8ac046ac56ebcde9ce08714e4cbf2d6942a/userdata/shm DeviceMajor:0 DeviceMinor:364 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:421 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-452 DeviceMajor:0 DeviceMinor:452 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:411 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:347 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:424 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-454 DeviceMajor:0 DeviceMinor:454 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:349 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:356 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fdf5f020-799f-4fc6-86c2-27b76e7bb567/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:409 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:347 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e23751d7-2d21-4805-8a90-fd6276de3545/volumes/kubernetes.io~projected/kube-api-access-z4b99 DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e12a94578f03e5c94a708a1f65d538b8641c5615f95421e15c5406b1b7a952cf/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bb27ebef79a610f4e6e4143ebc4253f4a644cfd67a8799f63d18b90f9bbab747/userdata/shm DeviceMajor:0 DeviceMinor:312 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:361 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-382 DeviceMajor:0 DeviceMinor:382 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-434 DeviceMajor:0 DeviceMinor:434 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8224a234-a815-4d78-9960-1c226c1224e4/volumes/kubernetes.io~projected/kube-api-access-84ntf DeviceMajor:0 DeviceMinor:52 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ced26a1d46e50a01617506a794137dc1eef61adb965bbc11eee56429b097313a/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/71604d9195264339523ac2dbfb9f5b3ea6ab2630c51ce057100d9e7e7e0e6a2e/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4dcb576a5d7db4d871f30cff4869ae4b04f4b2a68da30d8557f3c8eb09076577/userdata/shm DeviceMajor:0 DeviceMinor:413 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/volumes/kubernetes.io~projected/kube-api-access-pl9k5 DeviceMajor:0 DeviceMinor:425 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-447 DeviceMajor:0 DeviceMinor:447 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d1a59846-fdda-4740-ada7-13de6a939e93/volumes/kubernetes.io~projected/kube-api-access-qt65s DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe2359dc-5156-4bd5-8e7f-1bce48d948e6/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:285 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b176117b104c6ba75d453da4c5f021bd3fca01295c2ee5af4a5734afce77905d/userdata/shm DeviceMajor:0 DeviceMinor:287 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:358 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8b042ce3-beb3-4075-8360-9849305e7e83/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:344 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:346 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe7b8332-a94f-40a0-b4b5-5880f10b571a/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:402 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/2238e566-a39a-47ff-8f14-a27a404bc255/volumes/kubernetes.io~projected/kube-api-access-c9pvw DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4a5786b3-e466-4fc8-99c6-bc2876267401/volumes/kubernetes.io~projected/kube-api-access-gsn2n DeviceMajor:0 DeviceMinor:286 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/99ddf84f-2164-4309-a9bb-30a816909083/volumes/kubernetes.io~projected/kube-api-access-nwkmt DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/06846bf4-72b9-4348-b4ae-5108221dbd0b/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0eb207a7-1d34-4edf-8e0a-488b4363e175/volumes/kubernetes.io~projected/kube-api-access-swj72 DeviceMajor:0 DeviceMinor:309 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:343 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:355 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/101b3b78fdd561d363adeae439f500559787c01a606ed10d82262eb0e5bb4c88/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9cf3b1a7-af00-46ce-a1fc-70b02516a53f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:302 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e75fd1f7-4fef-489b-ab98-45bbaf194bdf/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:354 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d60eaa159a472139d1e983915b0ee2aaaa5faff5a84b30404dbc2d8457c12066/userdata/shm DeviceMajor:0 DeviceMinor:372 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/99dd82d0-8116-423a-8265-4c5b2f428ab8/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/7535ee53-f698-47f2-83fa-e0694379654a/volumes/kubernetes.io~secret/tls-certificates DeviceMajor:0 DeviceMinor:247 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0983ffbaeae93fc MacAddress:1e:2a:dd:3d:77:37 Speed:10000 Mtu:8901} {Name:2dea41af0a258a3 MacAddress:a2:67:5a:49:74:b6 Speed:10000 Mtu:8901} {Name:340a3c8fdb92d2b MacAddress:6e:a5:03:8e:e3:b2 Speed:10000 Mtu:8901} {Name:3b894f85b2fe38d MacAddress:06:cc:40:99:c0:11 Speed:10000 Mtu:8901} {Name:4dcb576a5d7db4d MacAddress:0a:f8:ff:0e:82:10 Speed:10000 Mtu:8901} {Name:5a5a34150e3e7a1 MacAddress:c6:f4:60:5c:8a:f1 Speed:10000 Mtu:8901} {Name:731e40fc0bbacca MacAddress:66:fc:77:07:d3:d0 Speed:10000 Mtu:8901} {Name:8606df236e3ab41 MacAddress:16:9c:6e:15:ce:c4 Speed:10000 Mtu:8901} {Name:b176117b104c6ba MacAddress:4a:99:ad:4b:fb:e9 Speed:10000 Mtu:8901} {Name:b6831c3f2c9e20d MacAddress:aa:9c:3f:6a:0d:12 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:41:0b:b5:7b:8f Speed:0 Mtu:9001} {Name:br-int MacAddress:ee:f8:0c:35:45:b1 Speed:0 Mtu:8901} {Name:cb1e06307a77553 MacAddress:f6:cf:58:10:67:10 Speed:10000 Mtu:8901} {Name:cc186bbec82ab8f MacAddress:0a:84:25:20:27:a6 Speed:10000 Mtu:8901} {Name:d226dc6231c66dd MacAddress:4e:77:c1:88:3d:36 Speed:10000 Mtu:8901} {Name:d3fa2a371f2d603 MacAddress:ea:67:a6:36:22:73 Speed:10000 Mtu:8901} {Name:d60eaa159a47213 MacAddress:e6:d0:40:74:45:46 Speed:10000 Mtu:8901} {Name:e87150dd997f161 MacAddress:8a:83:9e:16:e4:5a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:41:0b:b5:7b:8f Speed:0 Mtu:9001} {Name:f62229a1a7ae4d0 MacAddress:fa:85:4e:d8:b2:73 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:0a:5b:6d:e0:d0:3f Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:12:d7:5c:97:90:7f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:12:24.262454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.262459    5786 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:12:24.262910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.262534    5786 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:12:24.262996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.262943    5786 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:12:24.263195 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.262985    5786 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-134.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:12:24.263346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.263216    5786 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:12:24.263346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.263229    5786 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:12:24.263346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.263248    5786 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:12:24.263346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.263279    5786 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:12:24.263584 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.263429    5786 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:12:24.263773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.263757    5786 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:12:24.263960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.263944    5786 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:12:24.264027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.263985    5786 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:12:24.264027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.264000    5786 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:12:24.264027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.264009    5786 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:12:24.264027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.264020    5786 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:12:24.264301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.264034    5786 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:12:24.267412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.266274    5786 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:12:24.267412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.266295    5786 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:12:24.267412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.267052    5786 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:12:24.290506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.290475    5786 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:12:24.290506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.290500    5786 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:12:24.290710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.290640    5786 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:12:24.290949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.290929    5786 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:12:24.291031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.290978    5786 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:12:24.291118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291106    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:12:24.291179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291124    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:12:24.291179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291131    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:12:24.291179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291139    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:12:24.291179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291148    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:12:24.291179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291157    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:12:24.291179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291167    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:12:24.291179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291175    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:12:24.291179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291183    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:12:24.291532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291189    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:12:24.291532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291198    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:12:24.291532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291226    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:12:24.291532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291255    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:12:24.291532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.291261    5786 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:12:24.295636 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.295621    5786 server.go:1263] "Started kubelet"
Oct 30 21:12:24.296163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.296132    5786 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:12:24.296269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.296129    5786 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:12:24.296269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.296254    5786 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:12:24.296277 ip-10-0-132-134 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:12:24.296625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.296609    5786 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:12:24.297612 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.297597    5786 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.300104    5786 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.300128    5786 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.300258    5786 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.300270    5786 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.300296    5786 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:06:08 +0000 UTC" deadline="2028-04-23 00:03:08.133656449 +0000 UTC"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.300337    5786 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12962h50m43.833322556s"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.300361    5786 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.301060    5786 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-134.us-west-1.compute.internal\" not found"
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.304784    5786 factory.go:55] Registering systemd factory
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.304806    5786 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.305165    5786 factory.go:153] Registering CRI-O factory
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.305180    5786 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.305261    5786 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.305279    5786 factory.go:103] Registering Raw factory
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.305300    5786 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:12:24.306873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.305719    5786 manager.go:319] Starting recovery of all containers
Oct 30 21:12:24.313390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.313347    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2007f2be-74e2-4db5-ba7e-c3e2596777d6" volumeName="kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp" seLinuxMountContext=""
Oct 30 21:12:24.313577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.313554    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume" seLinuxMountContext=""
Oct 30 21:12:24.313691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.313674    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 21:12:24.313860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.313841    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c7d804a-c2a5-4923-8a9b-ed2803fdd02a" volumeName="kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client" seLinuxMountContext=""
Oct 30 21:12:24.313972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.313956    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99dd82d0-8116-423a-8265-4c5b2f428ab8" volumeName="kubernetes.io/secret/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:12:24.314085 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314066    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cf3b1a7-af00-46ce-a1fc-70b02516a53f" volumeName="kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:12:24.314151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314091    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eb207a7-1d34-4edf-8e0a-488b4363e175" volumeName="kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:12:24.314151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314114    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1daa2700-153a-4fed-8434-669dd9b021bb" volumeName="kubernetes.io/projected/1daa2700-153a-4fed-8434-669dd9b021bb-kube-api-access-68rr5" seLinuxMountContext=""
Oct 30 21:12:24.314301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314260    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2007f2be-74e2-4db5-ba7e-c3e2596777d6" volumeName="kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert" seLinuxMountContext=""
Oct 30 21:12:24.314301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314285    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55bed0f4-2cfe-4624-90b8-2f56b728d578" volumeName="kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 21:12:24.314417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314302    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b042ce3-beb3-4075-8360-9849305e7e83" volumeName="kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-tmp" seLinuxMountContext=""
Oct 30 21:12:24.314417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314317    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99dd82d0-8116-423a-8265-4c5b2f428ab8" volumeName="kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-env-overrides" seLinuxMountContext=""
Oct 30 21:12:24.314417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314340    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets" seLinuxMountContext=""
Oct 30 21:12:24.314417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314355    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/projected/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-kube-api-access-sc2mr" seLinuxMountContext=""
Oct 30 21:12:24.314417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314384    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99ddf84f-2164-4309-a9bb-30a816909083" volumeName="kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:12:24.314417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314402    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06846bf4-72b9-4348-b4ae-5108221dbd0b" volumeName="kubernetes.io/secret/06846bf4-72b9-4348-b4ae-5108221dbd0b-agent-certs" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314428    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eb207a7-1d34-4edf-8e0a-488b4363e175" volumeName="kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314445    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eb207a7-1d34-4edf-8e0a-488b4363e175" volumeName="kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314503    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2238e566-a39a-47ff-8f14-a27a404bc255" volumeName="kubernetes.io/projected/2238e566-a39a-47ff-8f14-a27a404bc255-kube-api-access-c9pvw" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314527    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c7d804a-c2a5-4923-8a9b-ed2803fdd02a" volumeName="kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314543    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55bed0f4-2cfe-4624-90b8-2f56b728d578" volumeName="kubernetes.io/projected/55bed0f4-2cfe-4624-90b8-2f56b728d578-kube-api-access-kjbpj" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314582    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314603    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2238e566-a39a-47ff-8f14-a27a404bc255" volumeName="kubernetes.io/secret/2238e566-a39a-47ff-8f14-a27a404bc255-cert" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314619    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c7d804a-c2a5-4923-8a9b-ed2803fdd02a" volumeName="kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314638    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b61fec4-1b8d-449c-b251-b48c3306d351" volumeName="kubernetes.io/projected/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-api-access-9mbqj" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314650    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7535ee53-f698-47f2-83fa-e0694379654a" volumeName="kubernetes.io/secret/7535ee53-f698-47f2-83fa-e0694379654a-tls-certificates" seLinuxMountContext=""
Oct 30 21:12:24.314685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314672    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b042ce3-beb3-4075-8360-9849305e7e83" volumeName="kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-etc-tuned" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314689    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314724    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e23751d7-2d21-4805-8a90-fd6276de3545" volumeName="kubernetes.io/projected/e23751d7-2d21-4805-8a90-fd6276de3545-kube-api-access-z4b99" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314740    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314758    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c7d804a-c2a5-4923-8a9b-ed2803fdd02a" volumeName="kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314771    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b61fec4-1b8d-449c-b251-b48c3306d351" volumeName="kubernetes.io/empty-dir/6b61fec4-1b8d-449c-b251-b48c3306d351-data-volume" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314790    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99dd82d0-8116-423a-8265-4c5b2f428ab8" volumeName="kubernetes.io/projected/99dd82d0-8116-423a-8265-4c5b2f428ab8-kube-api-access-dmf5z" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314806    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99ddf84f-2164-4309-a9bb-30a816909083" volumeName="kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314821    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314841    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1a59846-fdda-4740-ada7-13de6a939e93" volumeName="kubernetes.io/projected/d1a59846-fdda-4740-ada7-13de6a939e93-kube-api-access-qt65s" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314855    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314876    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f804883e-c5eb-46e0-a6d2-64db96efa860" volumeName="kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314890    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06846bf4-72b9-4348-b4ae-5108221dbd0b" volumeName="kubernetes.io/configmap/06846bf4-72b9-4348-b4ae-5108221dbd0b-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314906    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eb207a7-1d34-4edf-8e0a-488b4363e175" volumeName="kubernetes.io/empty-dir/0eb207a7-1d34-4edf-8e0a-488b4363e175-volume-directive-shadow" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314924    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2007f2be-74e2-4db5-ba7e-c3e2596777d6" volumeName="kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314937    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2007f2be-74e2-4db5-ba7e-c3e2596777d6" volumeName="kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314955    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8224a234-a815-4d78-9960-1c226c1224e4" volumeName="kubernetes.io/projected/8224a234-a815-4d78-9960-1c226c1224e4-kube-api-access-84ntf" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.314998    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cf3b1a7-af00-46ce-a1fc-70b02516a53f" volumeName="kubernetes.io/empty-dir/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-textfile" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315016    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cf3b1a7-af00-46ce-a1fc-70b02516a53f" volumeName="kubernetes.io/projected/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-kube-api-access-n6kcg" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315054    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315072    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99ddf84f-2164-4309-a9bb-30a816909083" volumeName="kubernetes.io/projected/99ddf84f-2164-4309-a9bb-30a816909083-kube-api-access-nwkmt" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315092    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cf3b1a7-af00-46ce-a1fc-70b02516a53f" volumeName="kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315126    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cf3b1a7-af00-46ce-a1fc-70b02516a53f" volumeName="kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-tls" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315163    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8b7c30e-9edc-43f6-ba65-279d40a3267e" volumeName="kubernetes.io/configmap/b8b7c30e-9edc-43f6-ba65-279d40a3267e-config-volume" seLinuxMountContext=""
Oct 30 21:12:24.315245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315186    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:12:24.315990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315740    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 21:12:24.315990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315773    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/empty-dir/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-tmp" seLinuxMountContext=""
Oct 30 21:12:24.315990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315790    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:12:24.315990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315811    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eb207a7-1d34-4edf-8e0a-488b4363e175" volumeName="kubernetes.io/projected/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-api-access-swj72" seLinuxMountContext=""
Oct 30 21:12:24.315990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315825    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a5786b3-e466-4fc8-99c6-bc2876267401" volumeName="kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n" seLinuxMountContext=""
Oct 30 21:12:24.315990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315917    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92966fe2-fe0f-494f-9bc8-87000dd086c9" volumeName="kubernetes.io/configmap/92966fe2-fe0f-494f-9bc8-87000dd086c9-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:12:24.315990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315972    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92966fe2-fe0f-494f-9bc8-87000dd086c9" volumeName="kubernetes.io/projected/92966fe2-fe0f-494f-9bc8-87000dd086c9-kube-api-access-vzngr" seLinuxMountContext=""
Oct 30 21:12:24.315990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.315989    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf5f020-799f-4fc6-86c2-27b76e7bb567" volumeName="kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316005    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf5f020-799f-4fc6-86c2-27b76e7bb567" volumeName="kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316019    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf5f020-799f-4fc6-86c2-27b76e7bb567" volumeName="kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-client-ca-bundle" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316035    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe2359dc-5156-4bd5-8e7f-1bce48d948e6" volumeName="kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316053    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99dd82d0-8116-423a-8265-4c5b2f428ab8" volumeName="kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316066    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1a59846-fdda-4740-ada7-13de6a939e93" volumeName="kubernetes.io/configmap/d1a59846-fdda-4740-ada7-13de6a939e93-serviceca" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316100    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316134    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f804883e-c5eb-46e0-a6d2-64db96efa860" volumeName="kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316148    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf5f020-799f-4fc6-86c2-27b76e7bb567" volumeName="kubernetes.io/projected/fdf5f020-799f-4fc6-86c2-27b76e7bb567-kube-api-access-v8vsb" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316156    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf5f020-799f-4fc6-86c2-27b76e7bb567" volumeName="kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 21:12:24.316358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316181    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe2359dc-5156-4bd5-8e7f-1bce48d948e6" volumeName="kubernetes.io/projected/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-kube-api-access-nssz5" seLinuxMountContext=""
Oct 30 21:12:24.316839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.316600    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe7b8332-a94f-40a0-b4b5-5880f10b571a" volumeName="kubernetes.io/secret/fe7b8332-a94f-40a0-b4b5-5880f10b571a-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 21:12:24.317239 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317193    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2007f2be-74e2-4db5-ba7e-c3e2596777d6" volumeName="kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config" seLinuxMountContext=""
Oct 30 21:12:24.317365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317345    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c7d804a-c2a5-4923-8a9b-ed2803fdd02a" volumeName="kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 21:12:24.317472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317365    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99dd82d0-8116-423a-8265-4c5b2f428ab8" volumeName="kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-config" seLinuxMountContext=""
Oct 30 21:12:24.317472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317379    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out" seLinuxMountContext=""
Oct 30 21:12:24.317472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317392    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/configmap/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:12:24.317472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317415    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f804883e-c5eb-46e0-a6d2-64db96efa860" volumeName="kubernetes.io/projected/f804883e-c5eb-46e0-a6d2-64db96efa860-kube-api-access-zh9l9" seLinuxMountContext=""
Oct 30 21:12:24.317472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317427    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2007f2be-74e2-4db5-ba7e-c3e2596777d6" volumeName="kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca" seLinuxMountContext=""
Oct 30 21:12:24.317472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317438    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55bed0f4-2cfe-4624-90b8-2f56b728d578" volumeName="kubernetes.io/configmap/55bed0f4-2cfe-4624-90b8-2f56b728d578-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:12:24.317472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317453    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55bed0f4-2cfe-4624-90b8-2f56b728d578" volumeName="kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:12:24.317472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317473    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317486    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e75fd1f7-4fef-489b-ab98-45bbaf194bdf" volumeName="kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317498    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f804883e-c5eb-46e0-a6d2-64db96efa860" volumeName="kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317509    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0492a73c-e41b-46c7-ad57-1f8220dfcf3a" volumeName="kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317536    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2007f2be-74e2-4db5-ba7e-c3e2596777d6" volumeName="kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317548    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317560    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf5f020-799f-4fc6-86c2-27b76e7bb567" volumeName="kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317572    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe7b8332-a94f-40a0-b4b5-5880f10b571a" volumeName="kubernetes.io/projected/fe7b8332-a94f-40a0-b4b5-5880f10b571a-kube-api-access-hlk7x" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317591    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e4cd6ee-1f05-473a-b32c-d9a53a790541" volumeName="kubernetes.io/projected/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kube-api-access-px5f9" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317603    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b61fec4-1b8d-449c-b251-b48c3306d351" volumeName="kubernetes.io/secret/6b61fec4-1b8d-449c-b251-b48c3306d351-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317617    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92966fe2-fe0f-494f-9bc8-87000dd086c9" volumeName="kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317628    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8b7c30e-9edc-43f6-ba65-279d40a3267e" volumeName="kubernetes.io/secret/b8b7c30e-9edc-43f6-ba65-279d40a3267e-metrics-tls" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317640    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317652    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e23751d7-2d21-4805-8a90-fd6276de3545" volumeName="kubernetes.io/empty-dir/e23751d7-2d21-4805-8a90-fd6276de3545-tmp-dir" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317664    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eb207a7-1d34-4edf-8e0a-488b4363e175" volumeName="kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317675    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c7d804a-c2a5-4923-8a9b-ed2803fdd02a" volumeName="kubernetes.io/projected/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-kube-api-access-pl9k5" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317689    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c7d804a-c2a5-4923-8a9b-ed2803fdd02a" volumeName="kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-client-tls" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317702    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b61fec4-1b8d-449c-b251-b48c3306d351" volumeName="kubernetes.io/configmap/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317713    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8224a234-a815-4d78-9960-1c226c1224e4" volumeName="kubernetes.io/configmap/8224a234-a815-4d78-9960-1c226c1224e4-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:12:24.317715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317723    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b042ce3-beb3-4075-8360-9849305e7e83" volumeName="kubernetes.io/projected/8b042ce3-beb3-4075-8360-9849305e7e83-kube-api-access-wxd9w" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317735    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92966fe2-fe0f-494f-9bc8-87000dd086c9" volumeName="kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317747    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cf3b1a7-af00-46ce-a1fc-70b02516a53f" volumeName="kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317764    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c7d804a-c2a5-4923-8a9b-ed2803fdd02a" volumeName="kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-federate-client-tls" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317791    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8b7c30e-9edc-43f6-ba65-279d40a3267e" volumeName="kubernetes.io/empty-dir/b8b7c30e-9edc-43f6-ba65-279d40a3267e-tmp-dir" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317804    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8b7c30e-9edc-43f6-ba65-279d40a3267e" volumeName="kubernetes.io/projected/b8b7c30e-9edc-43f6-ba65-279d40a3267e-kube-api-access-9w9zx" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317817    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2f2767-292c-45d1-a573-fb2749f866fe" volumeName="kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317826    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf5f020-799f-4fc6-86c2-27b76e7bb567" volumeName="kubernetes.io/empty-dir/fdf5f020-799f-4fc6-86c2-27b76e7bb567-audit-log" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317840    5786 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1daa2700-153a-4fed-8434-669dd9b021bb" volumeName="kubernetes.io/empty-dir/1daa2700-153a-4fed-8434-669dd9b021bb-tmp" seLinuxMountContext=""
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317850    5786 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:12:24.318407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.317856    5786 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:12:24.348092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.348052    5786 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:12:24.350896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.350864    5786 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:12:24.351181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.351169    5786 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:12:24.351322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.351309    5786 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:12:24.351460 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.351431    5786 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:12:24.405330 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.405280    5786 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-134.us-west-1.compute.internal\" not found"
Oct 30 21:12:24.413787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.413407    5786 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="5542bd59062a1711cc5fab143cec96e13e8a35145fe5fb9018b50387854b5bdf" exitCode=0
Oct 30 21:12:24.413787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.413441    5786 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="2d09aa7691466b9fc3475ff1c96a862f0bd19346c38165492a22cf6e8d491231" exitCode=0
Oct 30 21:12:24.413787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.413485    5786 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="316cccf27fb06dacfd9f62ed86981bc0c9502fc72e7e2de9c5c138454eb5ebde" exitCode=0
Oct 30 21:12:24.413787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.413496    5786 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="40d9d39791eab5574daf77751e02c5530f0ac0a5e47477cb90c9c85c6aeaa459" exitCode=0
Oct 30 21:12:24.413787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.413504    5786 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="f94ee1485369c6f86ba6c4c533ea02e208bc07e5f7606baf5b9ffe52c62bb2d8" exitCode=0
Oct 30 21:12:24.413787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.413512    5786 generic.go:356] "Generic (PLEG): container finished" podID="f804883e-c5eb-46e0-a6d2-64db96efa860" containerID="799dd8e44f7cff32935c1786adedf1d6ecd0f7f7e6313a66943acaee1e9712e8" exitCode=0
Oct 30 21:12:24.427065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.427032    5786 generic.go:356] "Generic (PLEG): container finished" podID="f9f2515e59c26a5d9f3c7f3a1c0a82c9" containerID="99737064de0e9db30413d2a7ec605a1e581f4b4428580e4e187bfe5163fbffa6" exitCode=0
Oct 30 21:12:24.451525 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.451493    5786 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:12:24.474540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.474476    5786 manager.go:324] Recovery completed
Oct 30 21:12:24.481620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.481287    5786 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2" exitCode=0
Oct 30 21:12:24.486172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.486146    5786 generic.go:356] "Generic (PLEG): container finished" podID="9cf3b1a7-af00-46ce-a1fc-70b02516a53f" containerID="7328fb709033dbf0cdfa174de2a1d812e5094047a7104e9138bed1465f531146" exitCode=0
Oct 30 21:12:24.490073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.489760    5786 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:12:24.490073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.490044    5786 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:12:24.490881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.490778    5786 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:12:24.501715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.501687    5786 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:12:24.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.503694    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:12:24.503855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.503737    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:12:24.503855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.503755    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:12:24.505380 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.505355    5786 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-134.us-west-1.compute.internal\" not found"
Oct 30 21:12:24.507227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507183    5786 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:12:24.507227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507200    5786 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:12:24.507375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507240    5786 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:12:24.507429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507392    5786 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:12:24.507429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507413    5786 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:12:24.507529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507432    5786 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:12:24.507529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507438    5786 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:12:24.507529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507444    5786 policy_none.go:50] "Start"
Oct 30 21:12:24.507529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507454    5786 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:12:24.507529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507468    5786 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:12:24.507859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.507840    5786 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:12:24.508488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.508444    5786 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:12:24.508488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.508467    5786 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:12:24.508488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.508477    5786 policy_none.go:44] "Start"
Oct 30 21:12:24.509329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.509309    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:12:24.509487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.509342    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:12:24.509487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.509357    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:12:24.516604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.515785    5786 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:12:24.520797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.520773    5786 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:12:24.520944 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.520814    5786 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:12:24.520944 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.520827    5786 server.go:87] "Starting device plugin registration server"
Oct 30 21:12:24.521225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.521182    5786 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:12:24.521282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.521232    5786 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:12:24.523248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.523227    5786 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:12:24.523365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.523331    5786 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:12:24.523419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.523375    5786 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:12:24.523419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.523382    5786 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:12:24.523538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.523458    5786 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:12:24.523364757Z","UUID":"dccbe5e2-6936-4215-a4a4-b4970461865a","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:12:24.525474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.525437    5786 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:12:24.525474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.525476    5786 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:12:24.529486 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.529446    5786 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:12:24.529587 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.529516    5786 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-134.us-west-1.compute.internal\" not found"
Oct 30 21:12:24.540056 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.540024    5786 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:12:24.587807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.587768    5786 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:12:24.621375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.621332    5786 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:12:24.622658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.622630    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:12:24.622782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.622674    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:12:24.622782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.622690    5786 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-134.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:12:24.622782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.622759    5786 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.652459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.652406    5786 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"]
Oct 30 21:12:24.653245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.653197    5786 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.653375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.653297    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"f9f2515e59c26a5d9f3c7f3a1c0a82c9","Type":"ContainerStarted","Data":"240457ed84b22bb167b212ea428802c7ae68e1d45b5cff3fed79e7b5722eb43d"}
Oct 30 21:12:24.653375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.653345    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"f9f2515e59c26a5d9f3c7f3a1c0a82c9","Type":"ContainerDied","Data":"99737064de0e9db30413d2a7ec605a1e581f4b4428580e4e187bfe5163fbffa6"}
Oct 30 21:12:24.653375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.653364    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"f9f2515e59c26a5d9f3c7f3a1c0a82c9","Type":"ContainerStarted","Data":"ced26a1d46e50a01617506a794137dc1eef61adb965bbc11eee56429b097313a"}
Oct 30 21:12:24.653497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.653414    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"d2a11b2240e0ab276a762b4e71b7c25e","Type":"ContainerStarted","Data":"d64bb761b6211c452c070f28bf0af5cfc74c99b19a6b0d2ed6011a759157f647"}
Oct 30 21:12:24.653497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.653418    5786 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.653497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.653431    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal" event={"ID":"d2a11b2240e0ab276a762b4e71b7c25e","Type":"ContainerStarted","Data":"77a7dbfc9efa8f6193edf5e13551c0fb5315d9c0570130c9f44cf06d07323780"}
Oct 30 21:12:24.720143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.720102    5786 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:12:24.720349 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.720170    5786 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.722921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.722898    5786 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:12:24.723061 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:24.722943    5786 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.736568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.736491    5786 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.736711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.736581    5786 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.791574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.791530    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.791760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.791587    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.791760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.791658    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d2a11b2240e0ab276a762b4e71b7c25e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"d2a11b2240e0ab276a762b4e71b7c25e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.891847 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.891802    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.892003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.891875    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.892003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.891913    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.892003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.891877    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f2515e59c26a5d9f3c7f3a1c0a82c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"f9f2515e59c26a5d9f3c7f3a1c0a82c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.892003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.891960    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d2a11b2240e0ab276a762b4e71b7c25e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"d2a11b2240e0ab276a762b4e71b7c25e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:24.892149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:24.892005    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d2a11b2240e0ab276a762b4e71b7c25e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal\" (UID: \"d2a11b2240e0ab276a762b4e71b7c25e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:25.264773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.264746    5786 apiserver.go:52] "Watching apiserver"
Oct 30 21:12:25.368543 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.368506    5786 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:12:25.369343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.369308    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-8295p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal","openshift-monitoring/metrics-server-7c86794bcf-ck4wn","openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6","openshift-ingress-canary/ingress-canary-fvn9q","openshift-network-diagnostics/network-check-target-c65g6","openshift-dns/node-resolver-nw865","openshift-multus/multus-47z7f","openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj","openshift-monitoring/node-exporter-5gr7z","openshift-console/console-75579f5f75-plqsn","openshift-monitoring/alertmanager-main-0","openshift-monitoring/thanos-querier-646bb56695-d6qr8","openshift-multus/network-metrics-daemon-nbcn2","openshift-monitoring/telemeter-client-78659679-f77rg","openshift-network-operator/iptables-alerter-k4zbv","openshift-image-registry/node-ca-mwrs5","openshift-ovn-kubernetes/ovnkube-node-4lnfl","openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz","openshift-cluster-node-tuning-operator/tuned-kk2jz","openshift-dns/dns-default-sjfxt","openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr","openshift-console/downloads-6dd789c8b7-b2mvs","kube-system/global-pull-secret-syncer-r59mn","kube-system/konnectivity-agent-zxr26","openshift-insights/insights-runtime-extractor-zwzcq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p","openshift-multus/multus-additional-cni-plugins-pttss"]
Oct 30 21:12:25.403392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.403356    5786 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:12:25.422443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422406    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.422443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422432    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.422734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422513    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.422734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422528    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.422734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422533    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xkwpv\""
Oct 30 21:12:25.422734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422542    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:12:25.422734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422606    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:12:25.422734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422614    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:12:25.422734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422630    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:12:25.422734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422688    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-swkpp\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422743    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422747    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wtj2v\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422756    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422769    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r8s65\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422791    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422813    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422858    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87wxx\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.422898    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.423146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.423040    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:12:25.496282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496249    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-wtmp\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.496464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496294    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.496464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496323    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.496464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496346    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-federate-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.496464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496397    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-systemd-units\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.496675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496494    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06846bf4-72b9-4348-b4ae-5108221dbd0b-agent-certs\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:12:25.496675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496567    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-serving-certs-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.496675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496606    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt65s\" (UniqueName: \"kubernetes.io/projected/d1a59846-fdda-4740-ada7-13de6a939e93-kube-api-access-qt65s\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:12:25.496675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496630    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.496675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496655    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-run\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.496893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496679    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-metrics-client-ca\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.496893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496704    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1a59846-fdda-4740-ada7-13de6a939e93-host\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:12:25.496893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496791    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.496893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496824    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxd9w\" (UniqueName: \"kubernetes.io/projected/8b042ce3-beb3-4075-8360-9849305e7e83-kube-api-access-wxd9w\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.496893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496851    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px5f9\" (UniqueName: \"kubernetes.io/projected/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kube-api-access-px5f9\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.496893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496880    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovn-node-metrics-cert\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496907    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496935    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b61fec4-1b8d-449c-b251-b48c3306d351-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496960    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.496986    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497064    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497105    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497131    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-dbus\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497152    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:25.497247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497194    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fdf5f020-799f-4fc6-86c2-27b76e7bb567-audit-log\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497277    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0eb207a7-1d34-4edf-8e0a-488b4363e175-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497304    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8224a234-a815-4d78-9960-1c226c1224e4-host-slash\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497325    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fdf5f020-799f-4fc6-86c2-27b76e7bb567-audit-log\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497345    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-hostroot\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497373    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0eb207a7-1d34-4edf-8e0a-488b4363e175-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497389    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-metrics-client-ca\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497422    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-tls\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497453    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b7c30e-9edc-43f6-ba65-279d40a3267e-config-volume\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497494    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nssz5\" (UniqueName: \"kubernetes.io/projected/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-kube-api-access-nssz5\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497513    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497540    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-grpc-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497563    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.497600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497590    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497609    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-metrics-server-audit-profiles\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497627    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b61fec4-1b8d-449c-b251-b48c3306d351-data-volume\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497654    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-bin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497680    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497726    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497750    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b61fec4-1b8d-449c-b251-b48c3306d351-data-volume\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497797    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497839    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-netns\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497868    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-log-socket\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497889    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-script-lib\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497913    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-conf\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497955    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6kcg\" (UniqueName: \"kubernetes.io/projected/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-kube-api-access-n6kcg\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.497983    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-multus-certs\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.498121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498078    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-cnibin\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498178    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-bin\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498231    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498253    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8224a234-a815-4d78-9960-1c226c1224e4-iptables-alerter-script\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498305    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-etc-kubernetes\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498494    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498533    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498553    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-client-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498572    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-tmp\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498595    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498620    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b7c30e-9edc-43f6-ba65-279d40a3267e-metrics-tls\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498640    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.498664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498656    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e23751d7-2d21-4805-8a90-fd6276de3545-hosts-file\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498679    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysconfig\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498704    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-systemd\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498731    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498736    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-tmp\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498778    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-cni-binary-copy\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498844    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-conf-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498894    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-registration-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498933    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498973    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-config\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498994    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-cni-binary-copy\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.498998    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499044    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe7b8332-a94f-40a0-b4b5-5880f10b571a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499077    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-etc-tuned\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499104    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mbqj\" (UniqueName: \"kubernetes.io/projected/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-api-access-9mbqj\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499123    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2238e566-a39a-47ff-8f14-a27a404bc255-cert\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:12:25.499164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499140    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499188    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgck7\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499239    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-modprobe-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499266    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9pvw\" (UniqueName: \"kubernetes.io/projected/2238e566-a39a-47ff-8f14-a27a404bc255-kube-api-access-c9pvw\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499292    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499315    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-slash\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499187    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b042ce3-beb3-4075-8360-9849305e7e83-etc-tuned\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499353    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499405    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499438    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-client-certs\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499477    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-lib-modules\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499507    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84ntf\" (UniqueName: \"kubernetes.io/projected/8224a234-a815-4d78-9960-1c226c1224e4-kube-api-access-84ntf\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499545    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499574    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-binary-copy\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499602    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-var-lib-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499628    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-tls\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499654    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-cnibin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499680    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-node-log\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499707    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-env-overrides\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499734    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499761    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-k8s-cni-cncf-io\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499788    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-netns\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499816    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8b7c30e-9edc-43f6-ba65-279d40a3267e-tmp-dir\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499832    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1a59846-fdda-4740-ada7-13de6a939e93-serviceca\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499828    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-cni-binary-copy\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.499920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499880    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc2mr\" (UniqueName: \"kubernetes.io/projected/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-kube-api-access-sc2mr\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499949    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.499957    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8b7c30e-9edc-43f6-ba65-279d40a3267e-tmp-dir\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500011    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1a59846-fdda-4740-ada7-13de6a939e93-serviceca\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500069    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500097    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500135    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e23751d7-2d21-4805-8a90-fd6276de3545-tmp-dir\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500160    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68rr5\" (UniqueName: \"kubernetes.io/projected/1daa2700-153a-4fed-8434-669dd9b021bb-kube-api-access-68rr5\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500183    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjbpj\" (UniqueName: \"kubernetes.io/projected/55bed0f4-2cfe-4624-90b8-2f56b728d578-kube-api-access-kjbpj\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500233    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e23751d7-2d21-4805-8a90-fd6276de3545-tmp-dir\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500248    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl9k5\" (UniqueName: \"kubernetes.io/projected/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-kube-api-access-pl9k5\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500266    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500296    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-device-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500325    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500353    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500401    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-systemd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500432    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-kubelet\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500505    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500547    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500631    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlk7x\" (UniqueName: \"kubernetes.io/projected/fe7b8332-a94f-40a0-b4b5-5880f10b571a-kube-api-access-hlk7x\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500653    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f804883e-c5eb-46e0-a6d2-64db96efa860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500683    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500709    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-multus\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500735    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-system-cni-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500751    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-metrics-certs\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500761    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500757    5786 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500799    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500850    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500877    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500934    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8vsb\" (UniqueName: \"kubernetes.io/projected/fdf5f020-799f-4fc6-86c2-27b76e7bb567-kube-api-access-v8vsb\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500942    5786 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500962    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-multus-daemon-config\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.501001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.500989    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501047    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501079    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-host\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501108    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501126    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-system-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501171    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501319    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-tmp\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501362    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501383    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-kubelet\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501433    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-kubelet-config\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501428    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-tmp\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501506    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-textfile\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501537    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501556    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh9l9\" (UniqueName: \"kubernetes.io/projected/f804883e-c5eb-46e0-a6d2-64db96efa860-kube-api-access-zh9l9\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501574    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7535ee53-f698-47f2-83fa-e0694379654a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-s47fz\" (UID: \"7535ee53-f698-47f2-83fa-e0694379654a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501626    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-sys\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501637    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-textfile\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501657    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501685    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-socket-dir-parent\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501723    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w9zx\" (UniqueName: \"kubernetes.io/projected/b8b7c30e-9edc-43f6-ba65-279d40a3267e-kube-api-access-9w9zx\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501818    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzngr\" (UniqueName: \"kubernetes.io/projected/92966fe2-fe0f-494f-9bc8-87000dd086c9-kube-api-access-vzngr\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501858    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swj72\" (UniqueName: \"kubernetes.io/projected/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-api-access-swj72\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501889    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501921    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-socket-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.501952    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4b99\" (UniqueName: \"kubernetes.io/projected/e23751d7-2d21-4805-8a90-fd6276de3545-kube-api-access-z4b99\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502026    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92966fe2-fe0f-494f-9bc8-87000dd086c9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502069    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55bed0f4-2cfe-4624-90b8-2f56b728d578-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502098    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502127    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502157    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-metrics-client-ca\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502185    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502239    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx5b5\" (UniqueName: \"kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502265    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-kubernetes\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502281    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-var-lib-kubelet\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502298    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-sys\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502315    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwkmt\" (UniqueName: \"kubernetes.io/projected/99ddf84f-2164-4309-a9bb-30a816909083-kube-api-access-nwkmt\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502330    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-etc-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.502491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502458    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmf5z\" (UniqueName: \"kubernetes.io/projected/99dd82d0-8116-423a-8265-4c5b2f428ab8-kube-api-access-dmf5z\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502516    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-root\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502622    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1daa2700-153a-4fed-8434-669dd9b021bb-tmp\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502676    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b61fec4-1b8d-449c-b251-b48c3306d351-crio-socket\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502711    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1daa2700-153a-4fed-8434-669dd9b021bb-tmp\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502743    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06846bf4-72b9-4348-b4ae-5108221dbd0b-konnectivity-ca\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502764    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-etc-selinux\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502816    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-sys-fs\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502869    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-ovn\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502916    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-netd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.503722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.502968    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.524343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.524194    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.524343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.524313    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.524589 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.524566    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.524723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.524661    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.525328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525303    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.525422 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525403    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:12:25.525483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525439    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:12:25.525638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525616    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:12:25.525879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525807    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:12:25.525879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525820    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.525879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525829    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:12:25.525879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525839    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.525879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525852    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.526164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.525975    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:12:25.526164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526009    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:12:25.526164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526015    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:12:25.526164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526008    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:12:25.526462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526445    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:12:25.526543 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526523    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 21:12:25.526543 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526532    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:12:25.526641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526526    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:12:25.526641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526560    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 21:12:25.526641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526533    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:12:25.526757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526731    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.526796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526760    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:12:25.526839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526825    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:12:25.526885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526853    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:12:25.526885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526857    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:12:25.526981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526936    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.526981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526958    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.527079 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.526998    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:12:25.527079 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527003    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:12:25.527079 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527040    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.527238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527085    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:12:25.527238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527140    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:12:25.527238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527165    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.527369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527299    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b61fec4-1b8d-449c-b251-b48c3306d351-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.527369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527338    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:12:25.527369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527340    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.527492 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527387    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:12:25.527538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527497    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:12:25.527585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527537    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:12:25.527585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527541    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.527585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527541    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:12:25.527585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527572    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 21:12:25.527585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527573    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:12:25.527804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527711    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:12:25.527804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527723    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:12:25.527804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527709    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-metrics-client-ca\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.527804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527765    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b7c30e-9edc-43f6-ba65-279d40a3267e-config-volume\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:25.527804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527773    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:12:25.528045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.527916    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.528097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528071    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:12:25.528097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528087    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-metrics-server-audit-profiles\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.528186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528106    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:12:25.528186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528107    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:12:25.528186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528130    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:12:25.528361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528187    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2fnqg\""
Oct 30 21:12:25.528361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528235    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 21:12:25.528361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528265    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:12:25.528361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528279    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-z8gbk\""
Oct 30 21:12:25.528361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528285    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:12:25.528361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528334    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-script-lib\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.528361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528351    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tklwz\""
Oct 30 21:12:25.528657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528362    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t4qh6\""
Oct 30 21:12:25.528657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528374    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dssbm\""
Oct 30 21:12:25.528657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528366    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-rvh9q\""
Oct 30 21:12:25.528657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528405    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:12:25.528657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528414    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 21:12:25.528657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528421    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-6sttw\""
Oct 30 21:12:25.528657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528429    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:12:25.528951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528793    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:12:25.528951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528826    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:12:25.528951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528949    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wxcc9\""
Oct 30 21:12:25.529068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528973    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:12:25.529116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529070    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a4l5rnn1lts2h\""
Oct 30 21:12:25.529286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529265    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zwgx7\""
Oct 30 21:12:25.529387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529363    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovnkube-config\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.529450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529395    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.529450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529402    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.529561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529476    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8224a234-a815-4d78-9960-1c226c1224e4-iptables-alerter-script\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:12:25.529642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529603    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdf5f020-799f-4fc6-86c2-27b76e7bb567-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.529698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.528949    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-x9kft\""
Oct 30 21:12:25.529698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529643    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2238e566-a39a-47ff-8f14-a27a404bc255-cert\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:12:25.529698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529664    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:25.529976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529835    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-k8b2g\""
Oct 30 21:12:25.529976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529864    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vwtq2\""
Oct 30 21:12:25.530081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.529996    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-client-certs\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.530132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530088    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.530132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530093    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cb6bb\""
Oct 30 21:12:25.530239 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530149    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99dd82d0-8116-423a-8265-4c5b2f428ab8-env-overrides\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.530239 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530189    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-secret-metrics-server-tls\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.530341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530195    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:12:25.530498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530412    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c2dlm\""
Oct 30 21:12:25.530498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530427    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.530613 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530578    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rc6nc\""
Oct 30 21:12:25.530613 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530585    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xrxgx\""
Oct 30 21:12:25.530711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530648    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.530868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530803    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vpcr6\""
Oct 30 21:12:25.530868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530815    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:12:25.530868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530857    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.531030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.530863    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.531156 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.531054    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.531233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.531189    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99ddf84f-2164-4309-a9bb-30a816909083-multus-daemon-config\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.531401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.531318    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 21:12:25.531401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.531364    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.531520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.531498    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.531628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.531534    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-secret-telemeter-client\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.531628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.531539    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533549    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533569    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533578    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-metrics-client-ca\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533575    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533576    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533609    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7535ee53-f698-47f2-83fa-e0694379654a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-s47fz\" (UID: \"7535ee53-f698-47f2-83fa-e0694379654a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533730    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06846bf4-72b9-4348-b4ae-5108221dbd0b-konnectivity-ca\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533773    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92966fe2-fe0f-494f-9bc8-87000dd086c9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533785    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533821    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55bed0f4-2cfe-4624-90b8-2f56b728d578-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533825    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.534095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.533970    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537398    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537409    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-metrics-client-ca\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537409    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537442    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99dd82d0-8116-423a-8265-4c5b2f428ab8-ovn-node-metrics-cert\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537444    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-serving-certs-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537498    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537515    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-federate-client-tls\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537588    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06846bf4-72b9-4348-b4ae-5108221dbd0b-agent-certs\") pod \"konnectivity-agent-zxr26\" (UID: \"06846bf4-72b9-4348-b4ae-5108221dbd0b\") " pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:12:25.537626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537593    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92966fe2-fe0f-494f-9bc8-87000dd086c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:25.538118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537683    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-tls\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.538118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.537879    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-grpc-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.538118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.538027    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.538118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.538030    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.540016 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.539985    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/55bed0f4-2cfe-4624-90b8-2f56b728d578-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:25.572131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572091    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:12:25.572603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572579    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:12:25.572785 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572768    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1fddjiuem6drv\""
Oct 30 21:12:25.572855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572792    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4fqz7\""
Oct 30 21:12:25.572855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572826    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vcb5m\""
Oct 30 21:12:25.572855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572841    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ljsb2\""
Oct 30 21:12:25.572999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572855    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pmmk6\""
Oct 30 21:12:25.572999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572875    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:12:25.572999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.572905    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:12:25.573229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.573193    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4k42l\""
Oct 30 21:12:25.573360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.573332    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:12:25.573558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.573542    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:12:25.573716 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.573620    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:12:25.576069 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.576037    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577158    5786 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577424    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-original-pull-secret\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577488    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68rr5\" (UniqueName: \"kubernetes.io/projected/1daa2700-153a-4fed-8434-669dd9b021bb-kube-api-access-68rr5\") pod \"downloads-6dd789c8b7-b2mvs\" (UID: \"1daa2700-153a-4fed-8434-669dd9b021bb\") " pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577506    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577561    5786 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577629    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nssz5\" (UniqueName: \"kubernetes.io/projected/fe2359dc-5156-4bd5-8e7f-1bce48d948e6-kube-api-access-nssz5\") pod \"network-metrics-daemon-nbcn2\" (UID: \"fe2359dc-5156-4bd5-8e7f-1bce48d948e6\") " pod="openshift-multus/network-metrics-daemon-nbcn2"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577692    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt65s\" (UniqueName: \"kubernetes.io/projected/d1a59846-fdda-4740-ada7-13de6a939e93-kube-api-access-qt65s\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:25.577761    5786 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577867    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh9l9\" (UniqueName: \"kubernetes.io/projected/f804883e-c5eb-46e0-a6d2-64db96efa860-kube-api-access-zh9l9\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.577973    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl9k5\" (UniqueName: \"kubernetes.io/projected/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-kube-api-access-pl9k5\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578108    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8vsb\" (UniqueName: \"kubernetes.io/projected/fdf5f020-799f-4fc6-86c2-27b76e7bb567-kube-api-access-v8vsb\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578145    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgck7\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578154    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-secret-thanos-querier-tls\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578200    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzngr\" (UniqueName: \"kubernetes.io/projected/92966fe2-fe0f-494f-9bc8-87000dd086c9-kube-api-access-vzngr\") pod \"openshift-state-metrics-7bd7489957-8295p\" (UID: \"92966fe2-fe0f-494f-9bc8-87000dd086c9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8295p"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578524    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlk7x\" (UniqueName: \"kubernetes.io/projected/fe7b8332-a94f-40a0-b4b5-5880f10b571a-kube-api-access-hlk7x\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:25.578586    5786 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-134.us-west-1.compute.internal"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578701    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxd9w\" (UniqueName: \"kubernetes.io/projected/8b042ce3-beb3-4075-8360-9849305e7e83-kube-api-access-wxd9w\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578707    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swj72\" (UniqueName: \"kubernetes.io/projected/0eb207a7-1d34-4edf-8e0a-488b4363e175-kube-api-access-swj72\") pod \"kube-state-metrics-5dc8d7d89b-x6plj\" (UID: \"0eb207a7-1d34-4edf-8e0a-488b4363e175\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-x6plj"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578856    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjbpj\" (UniqueName: \"kubernetes.io/projected/55bed0f4-2cfe-4624-90b8-2f56b728d578-kube-api-access-kjbpj\") pod \"prometheus-operator-9cd89b74c-6rvsr\" (UID: \"55bed0f4-2cfe-4624-90b8-2f56b728d578\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6rvsr"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578866    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578893    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9pvw\" (UniqueName: \"kubernetes.io/projected/2238e566-a39a-47ff-8f14-a27a404bc255-kube-api-access-c9pvw\") pod \"ingress-canary-fvn9q\" (UID: \"2238e566-a39a-47ff-8f14-a27a404bc255\") " pod="openshift-ingress-canary/ingress-canary-fvn9q"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578905    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc2mr\" (UniqueName: \"kubernetes.io/projected/e75fd1f7-4fef-489b-ab98-45bbaf194bdf-kube-api-access-sc2mr\") pod \"thanos-querier-646bb56695-d6qr8\" (UID: \"e75fd1f7-4fef-489b-ab98-45bbaf194bdf\") " pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579038    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b7c30e-9edc-43f6-ba65-279d40a3267e-metrics-tls\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.578858    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx5b5\" (UniqueName: \"kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5\") pod \"console-75579f5f75-plqsn\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") " pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579348    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84ntf\" (UniqueName: \"kubernetes.io/projected/8224a234-a815-4d78-9960-1c226c1224e4-kube-api-access-84ntf\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579377    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6kcg\" (UniqueName: \"kubernetes.io/projected/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-kube-api-access-n6kcg\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579398    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4b99\" (UniqueName: \"kubernetes.io/projected/e23751d7-2d21-4805-8a90-fd6276de3545-kube-api-access-z4b99\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579377    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwkmt\" (UniqueName: \"kubernetes.io/projected/99ddf84f-2164-4309-a9bb-30a816909083-kube-api-access-nwkmt\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579547    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579554    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe7b8332-a94f-40a0-b4b5-5880f10b571a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jp9t6\" (UID: \"fe7b8332-a94f-40a0-b4b5-5880f10b571a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579549    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w9zx\" (UniqueName: \"kubernetes.io/projected/b8b7c30e-9edc-43f6-ba65-279d40a3267e-kube-api-access-9w9zx\") pod \"dns-default-sjfxt\" (UID: \"b8b7c30e-9edc-43f6-ba65-279d40a3267e\") " pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579594    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdf5f020-799f-4fc6-86c2-27b76e7bb567-client-ca-bundle\") pod \"metrics-server-7c86794bcf-ck4wn\" (UID: \"fdf5f020-799f-4fc6-86c2-27b76e7bb567\") " pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579782    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmf5z\" (UniqueName: \"kubernetes.io/projected/99dd82d0-8116-423a-8265-4c5b2f428ab8-kube-api-access-dmf5z\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579783    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsn2n\" (UniqueName: \"kubernetes.io/projected/4a5786b3-e466-4fc8-99c6-bc2876267401-kube-api-access-gsn2n\") pod \"network-check-target-c65g6\" (UID: \"4a5786b3-e466-4fc8-99c6-bc2876267401\") " pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579827    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px5f9\" (UniqueName: \"kubernetes.io/projected/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kube-api-access-px5f9\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.579964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.579925    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mbqj\" (UniqueName: \"kubernetes.io/projected/6b61fec4-1b8d-449c-b251-b48c3306d351-kube-api-access-9mbqj\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.581382 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.580049    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c7d804a-c2a5-4923-8a9b-ed2803fdd02a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78659679-f77rg\" (UID: \"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a\") " pod="openshift-monitoring/telemeter-client-78659679-f77rg"
Oct 30 21:12:25.581903 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.581883    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.582105 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.582087    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:12:25.603644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603608    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-netns\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603662    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-device-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603686    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-systemd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603708    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-kubelet\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603722    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-netns\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603730    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-multus\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603772    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-kubelet\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603776    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-system-cni-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603788    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-device-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603776    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-systemd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603816    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-multus\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603834    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.603881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603849    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-system-cni-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603895    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603966    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.603995    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604000    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604052    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-host\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604055    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604074    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-system-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604103    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-kubelet\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604108    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-host\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604125    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-kubelet-config\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604146    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-sys\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604148    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-system-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604153    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-kubelet\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604195    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-sys\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604196    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-kubelet-config\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604192    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-socket-dir-parent\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604252    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-socket-dir-parent\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604251    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-socket-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604290    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-kubernetes\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604293    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-socket-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604317    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-var-lib-kubelet\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604340    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-kubernetes\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604383    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-var-lib-kubelet\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604340    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-sys\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604385    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-sys\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604434    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-etc-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604458    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-root\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604478    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b61fec4-1b8d-449c-b251-b48c3306d351-crio-socket\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.604500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604498    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-etc-selinux\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604588    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-etc-selinux\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604614    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-root\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604627    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-sys-fs\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604634    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-etc-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604651    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b61fec4-1b8d-449c-b251-b48c3306d351-crio-socket\") pod \"insights-runtime-extractor-zwzcq\" (UID: \"6b61fec4-1b8d-449c-b251-b48c3306d351\") " pod="openshift-insights/insights-runtime-extractor-zwzcq"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604729    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-ovn\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604749    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-sys-fs\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604761    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-netd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604786    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-wtmp\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604799    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-run-ovn\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604813    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604840    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-netd\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604852    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9cf3b1a7-af00-46ce-a1fc-70b02516a53f-node-exporter-wtmp\") pod \"node-exporter-5gr7z\" (UID: \"9cf3b1a7-af00-46ce-a1fc-70b02516a53f\") " pod="openshift-monitoring/node-exporter-5gr7z"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604866    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-systemd-units\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604897    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-systemd-units\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604924    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-run\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604944    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-cni-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604951    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1a59846-fdda-4740-ada7-13de6a939e93-host\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.604994    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1a59846-fdda-4740-ada7-13de6a939e93-host\") pod \"node-ca-mwrs5\" (UID: \"d1a59846-fdda-4740-ada7-13de6a939e93\") " pod="openshift-image-registry/node-ca-mwrs5"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605009    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605036    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-dbus\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605062    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8224a234-a815-4d78-9960-1c226c1224e4-host-slash\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605012    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-run\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605085    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-hostroot\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605117    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-bin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605122    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8224a234-a815-4d78-9960-1c226c1224e4-host-slash\") pod \"iptables-alerter-k4zbv\" (UID: \"8224a234-a815-4d78-9960-1c226c1224e4\") " pod="openshift-network-operator/iptables-alerter-k4zbv"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605128    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0492a73c-e41b-46c7-ad57-1f8220dfcf3a-dbus\") pod \"global-pull-secret-syncer-r59mn\" (UID: \"0492a73c-e41b-46c7-ad57-1f8220dfcf3a\") " pod="kube-system/global-pull-secret-syncer-r59mn"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605136    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605141    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-netns\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605171    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-hostroot\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605186    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-log-socket\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605230    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-run-netns\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605172    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-var-lib-cni-bin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605239    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-conf\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605270    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-multus-certs\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605296    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-multus-certs\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605307    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-log-socket\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605298    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-cnibin\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605325    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f804883e-c5eb-46e0-a6d2-64db96efa860-cnibin\") pod \"multus-additional-cni-plugins-pttss\" (UID: \"f804883e-c5eb-46e0-a6d2-64db96efa860\") " pod="openshift-multus/multus-additional-cni-plugins-pttss"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605346    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-bin\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605366    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-conf\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605368    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-etc-kubernetes\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605397    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-etc-kubernetes\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605404    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e23751d7-2d21-4805-8a90-fd6276de3545-hosts-file\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605408    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-cni-bin\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605429    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysconfig\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605474    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-systemd\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605514    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-conf-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605519    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e23751d7-2d21-4805-8a90-fd6276de3545-hosts-file\") pod \"node-resolver-nw865\" (UID: \"e23751d7-2d21-4805-8a90-fd6276de3545\") " pod="openshift-dns/node-resolver-nw865"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605533    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysconfig\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605541    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-registration-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605575    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-systemd\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605584    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605591    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-multus-conf-dir\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605599    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-registration-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605606    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-modprobe-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605643    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-slash\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605648    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e4cd6ee-1f05-473a-b32c-d9a53a790541-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f6z9p\" (UID: \"1e4cd6ee-1f05-473a-b32c-d9a53a790541\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f6z9p"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605675    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-lib-modules\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605687    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-host-slash\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605704    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-var-lib-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605704    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-modprobe-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605747    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-var-lib-openvswitch\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605778    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-cnibin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605797    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-lib-modules\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.605760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605802    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-node-log\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.608174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605836    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99dd82d0-8116-423a-8265-4c5b2f428ab8-node-log\") pod \"ovnkube-node-4lnfl\" (UID: \"99dd82d0-8116-423a-8265-4c5b2f428ab8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:25.608174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605836    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.608174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605846    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-cnibin\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.608174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605862    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-k8s-cni-cncf-io\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.608174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605922    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ddf84f-2164-4309-a9bb-30a816909083-host-run-k8s-cni-cncf-io\") pod \"multus-47z7f\" (UID: \"99ddf84f-2164-4309-a9bb-30a816909083\") " pod="openshift-multus/multus-47z7f"
Oct 30 21:12:25.608174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.605941    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b042ce3-beb3-4075-8360-9849305e7e83-etc-sysctl-d\") pod \"tuned-kk2jz\" (UID: \"8b042ce3-beb3-4075-8360-9849305e7e83\") " pod="openshift-cluster-node-tuning-operator/tuned-kk2jz"
Oct 30 21:12:25.680462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.680442    5786 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:12:25.688133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:25.688110    5786 scope.go:122] "RemoveContainer" containerID="8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9"
Oct 30 21:12:26.026419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:26.026385    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:12:26.509558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:26.509518    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerStarted","Data":"28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395"}
Oct 30 21:12:26.510013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:26.509610    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerStarted","Data":"984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d"}
Oct 30 21:12:26.510013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:26.509658    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerStarted","Data":"c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8"}
Oct 30 21:12:26.510013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:26.509696    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerStarted","Data":"9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317"}
Oct 30 21:12:26.697672 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:26.697636    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sjfxt"
Oct 30 21:12:27.409058 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.409031    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:27.416590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.416560    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8"
Oct 30 21:12:27.430839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.430815    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zxr26"
Oct 30 21:12:27.480757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.480730    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:27.517265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.517143    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8" event={"ID":"e75fd1f7-4fef-489b-ab98-45bbaf194bdf","Type":"ContainerStarted","Data":"a175a58b1601c14a07d815f2c5fbd0f4fa29d1910b44d9d20ba412dd4452a7d9"}
Oct 30 21:12:27.520508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.520482    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78659679-f77rg" event={"ID":"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a","Type":"ContainerStarted","Data":"490ff1b3aa6b61192bdde252ad166692874e084a926d5b5efccc31942d98e2bc"}
Oct 30 21:12:27.520632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.520551    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78659679-f77rg" event={"ID":"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a","Type":"ContainerStarted","Data":"7e41d883f55bfc7e9c97f4eae917cddc0cb80994936624aeae7d9e848080665b"}
Oct 30 21:12:27.520632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.520580    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78659679-f77rg" event={"ID":"2c7d804a-c2a5-4923-8a9b-ed2803fdd02a","Type":"ContainerStarted","Data":"4e8d7e64aa9069773ce027c0ee0824adafc554f89d5b062517bac23bd780e695"}
Oct 30 21:12:27.852385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.852352    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:27.852667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:27.852654    5786 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:12:28.121005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:28.120969    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:28.529654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:28.529611    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerStarted","Data":"f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2"}
Oct 30 21:12:28.533588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:28.533554    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8" event={"ID":"e75fd1f7-4fef-489b-ab98-45bbaf194bdf","Type":"ContainerStarted","Data":"e2b7614d96deec8054ce59b1a6f36c20913f5af47d513989c3b0e68196781115"}
Oct 30 21:12:28.533779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:28.533627    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-646bb56695-d6qr8" event={"ID":"e75fd1f7-4fef-489b-ab98-45bbaf194bdf","Type":"ContainerStarted","Data":"a67e8865f760f129eeacf990c84b8d798f51cf165f6cf843c0231fff8647e347"}
Oct 30 21:12:28.759243 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:28.759183    5786 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = unable to pull image: copying system image from manifest list: copying config: context canceled" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36c3da5821b9e51334e9c523aca7fca6a9ef8b853fd4f238b908b086b129619d"
Oct 30 21:12:28.759462 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:28.759439    5786 kuberuntime_manager.go:1707] "Unhandled Error" err="container download-server start failed in pod downloads-6dd789c8b7-b2mvs_openshift-console(1daa2700-153a-4fed-8434-669dd9b021bb): ErrImagePull: rpc error: code = Canceled desc = unable to pull image: copying system image from manifest list: copying config: context canceled" logger="UnhandledError"
Oct 30 21:12:28.759538 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:28.759483    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"download-server\" with ErrImagePull: \"rpc error: code = Canceled desc = unable to pull image: copying system image from manifest list: copying config: context canceled\"" pod="openshift-console/downloads-6dd789c8b7-b2mvs" podUID="1daa2700-153a-4fed-8434-669dd9b021bb"
Oct 30 21:12:28.822921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:28.822827    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:28.829622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:28.829596    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:29.496365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.496325    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:29.627001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.626967    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-585d545cdf-xtfgw"]
Oct 30 21:12:29.629115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.629096    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.644104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.644068    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-service-ca\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.644289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.644107    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-oauth-config\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.644289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.644126    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-oauth-serving-cert\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.644289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.644161    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64gc9\" (UniqueName: \"kubernetes.io/projected/60e1e587-2825-4594-9c22-16fdd5ab0336-kube-api-access-64gc9\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.644289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.644186    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-trusted-ca-bundle\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.644289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.644230    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-serving-cert\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.644568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.644345    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-console-config\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.644568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.644534    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e1e587-2825-4594-9c22-16fdd5ab0336-tmp\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.681585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.681553    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:12:29.688588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.688558    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-585d545cdf-xtfgw"]
Oct 30 21:12:29.746806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.746723    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-service-ca\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.746806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.746776    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-oauth-config\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.746805    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-oauth-serving-cert\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.746838    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64gc9\" (UniqueName: \"kubernetes.io/projected/60e1e587-2825-4594-9c22-16fdd5ab0336-kube-api-access-64gc9\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.746866    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-trusted-ca-bundle\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.746894    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-serving-cert\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.746963    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-console-config\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.746985    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e1e587-2825-4594-9c22-16fdd5ab0336-tmp\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747415 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.747233    5786 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:12:29.747468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.747422    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e1e587-2825-4594-9c22-16fdd5ab0336-tmp\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.747605    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-service-ca\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.747714 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.747623    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-oauth-serving-cert\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.748065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.748039    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-console-config\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.748815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.748785    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-trusted-ca-bundle\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.749789 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.749759    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-serving-cert\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.750124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.750107    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-oauth-config\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.803034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.803004    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64gc9\" (UniqueName: \"kubernetes.io/projected/60e1e587-2825-4594-9c22-16fdd5ab0336-kube-api-access-64gc9\") pod \"console-585d545cdf-xtfgw\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") " pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:29.939551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:29.939517    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:30.313038 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:30.313000    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-585d545cdf-xtfgw"]
Oct 30 21:12:30.316686 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:30.316652    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e1e587_2825_4594_9c22_16fdd5ab0336.slice/crio-29430b653651cb5282ad669e6cfbe4b898d20f48ad92270b2c50f8e817c2ea98 WatchSource:0}: Error finding container 29430b653651cb5282ad669e6cfbe4b898d20f48ad92270b2c50f8e817c2ea98: Status 404 returned error can't find the container with id 29430b653651cb5282ad669e6cfbe4b898d20f48ad92270b2c50f8e817c2ea98
Oct 30 21:12:30.500925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:30.500893    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-s47fz"
Oct 30 21:12:30.542619 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:30.542579    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-585d545cdf-xtfgw" event={"ID":"60e1e587-2825-4594-9c22-16fdd5ab0336","Type":"ContainerStarted","Data":"11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b"}
Oct 30 21:12:30.542881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:30.542633    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-585d545cdf-xtfgw" event={"ID":"60e1e587-2825-4594-9c22-16fdd5ab0336","Type":"ContainerStarted","Data":"29430b653651cb5282ad669e6cfbe4b898d20f48ad92270b2c50f8e817c2ea98"}
Oct 30 21:12:30.659996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:30.659884    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-585d545cdf-xtfgw" podStartSLOduration=1.659870102 podStartE2EDuration="1.659870102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:30.659257658 +0000 UTC m=+6.530376778" watchObservedRunningTime="2026-10-30 21:12:30.659870102 +0000 UTC m=+6.530989221"
Oct 30 21:12:31.206633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:31.206597    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:31.207898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:31.207874    5786 patch_prober.go:29] interesting pod/console-585d545cdf-xtfgw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" start-of-body=
Oct 30 21:12:31.208016 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:31.207918    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-console/console-585d545cdf-xtfgw" podUID="60e1e587-2825-4594-9c22-16fdd5ab0336" containerName="console" probeResult="failure" output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused"
Oct 30 21:12:33.688731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:33.688694    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:33.693449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:33.693428    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jp9t6"
Oct 30 21:12:37.991170 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:37.991135    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:39.501778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:39.501745    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:12:41.212551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:41.212521    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:41.216391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:41.216371    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:12:41.472164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:41.472084    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75579f5f75-plqsn"]
Oct 30 21:12:42.097173 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:42.097136    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:44.764100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:44.764070    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4lnfl"
Oct 30 21:12:45.606333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:45.606301    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-b2mvs" event={"ID":"1daa2700-153a-4fed-8434-669dd9b021bb","Type":"ContainerStarted","Data":"093c34d2792264e7dbd0d1b7a6f59df1e99726dc86ddbe14bf6d6a53d77b5257"}
Oct 30 21:12:45.606592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:45.606571    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:45.608285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:45.608258    5786 patch_prober.go:29] interesting pod/downloads-6dd789c8b7-b2mvs container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused" start-of-body=
Oct 30 21:12:45.608389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:45.608312    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6dd789c8b7-b2mvs" podUID="1daa2700-153a-4fed-8434-669dd9b021bb" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused"
Oct 30 21:12:46.623879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:46.623843    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-b2mvs"
Oct 30 21:12:48.126940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:48.126912    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:48.131336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:48.131311    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7c86794bcf-ck4wn"
Oct 30 21:12:52.377077 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:12:52.377034    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:12:54.194905 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.194869    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-64c49d8448-l4drv"]
Oct 30 21:12:54.227118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.227083    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.329327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.329292    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c49d8448-l4drv"]
Oct 30 21:12:54.374822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.374790    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-service-ca\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.375007 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.374830    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-oauth-config\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.375007 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.374860    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-serving-cert\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.375112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.375027    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-config\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.375112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.375064    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmkzc\" (UniqueName: \"kubernetes.io/projected/c435ad8a-42bd-4839-b96e-4320ca0d44d5-kube-api-access-bmkzc\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.375112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.375099    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-trusted-ca-bundle\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.375258 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.375235    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-oauth-serving-cert\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.375308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.375270    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c435ad8a-42bd-4839-b96e-4320ca0d44d5-tmp\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.476335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.476236    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-serving-cert\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.476335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.476286    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-config\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.476335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.476307    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmkzc\" (UniqueName: \"kubernetes.io/projected/c435ad8a-42bd-4839-b96e-4320ca0d44d5-kube-api-access-bmkzc\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.476632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.476344    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-trusted-ca-bundle\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.476632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.476396    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-oauth-serving-cert\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.476632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.476426    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c435ad8a-42bd-4839-b96e-4320ca0d44d5-tmp\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.476632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.476481    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-service-ca\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.476632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.476511    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-oauth-config\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.477182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.477154    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-config\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.477477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.477451    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-oauth-serving-cert\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.477618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.477495    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c435ad8a-42bd-4839-b96e-4320ca0d44d5-tmp\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.478003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.477978    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-service-ca\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.478465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.478443    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c435ad8a-42bd-4839-b96e-4320ca0d44d5-trusted-ca-bundle\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.479489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.479466    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-serving-cert\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.479590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.479494    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c435ad8a-42bd-4839-b96e-4320ca0d44d5-console-oauth-config\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.535240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.535190    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmkzc\" (UniqueName: \"kubernetes.io/projected/c435ad8a-42bd-4839-b96e-4320ca0d44d5-kube-api-access-bmkzc\") pod \"console-64c49d8448-l4drv\" (UID: \"c435ad8a-42bd-4839-b96e-4320ca0d44d5\") " pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.556708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.556678    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:12:54.962582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:54.962553    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c49d8448-l4drv"]
Oct 30 21:12:54.965928 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:12:54.965900    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc435ad8a_42bd_4839_b96e_4320ca0d44d5.slice/crio-441e23b686b28e47b99ba226fba308b507e58c34d523787b00aeae596be98240 WatchSource:0}: Error finding container 441e23b686b28e47b99ba226fba308b507e58c34d523787b00aeae596be98240: Status 404 returned error can't find the container with id 441e23b686b28e47b99ba226fba308b507e58c34d523787b00aeae596be98240
Oct 30 21:12:55.652614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:55.652580    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c49d8448-l4drv" event={"ID":"c435ad8a-42bd-4839-b96e-4320ca0d44d5","Type":"ContainerStarted","Data":"bc7e5c47f5276971df1d0df50a1542adc0409dfa3f459819ac805d332c6cf91c"}
Oct 30 21:12:55.653014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:55.652673    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c49d8448-l4drv" event={"ID":"c435ad8a-42bd-4839-b96e-4320ca0d44d5","Type":"ContainerStarted","Data":"441e23b686b28e47b99ba226fba308b507e58c34d523787b00aeae596be98240"}
Oct 30 21:12:55.794639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:55.794582    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-64c49d8448-l4drv" podStartSLOduration=1.794567341 podStartE2EDuration="1.794567341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:55.793872941 +0000 UTC m=+31.664992056" watchObservedRunningTime="2026-10-30 21:12:55.794567341 +0000 UTC m=+31.665686460"
Oct 30 21:12:56.512716 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:12:56.512689    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c65g6"
Oct 30 21:13:00.276029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.275990    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:13:00.276633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.276374    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy-web" containerID="cri-o://c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8" gracePeriod=120
Oct 30 21:13:00.276633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.276407    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy" containerID="cri-o://984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d" gracePeriod=120
Oct 30 21:13:00.276633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.276464    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="prom-label-proxy" containerID="cri-o://f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2" gracePeriod=120
Oct 30 21:13:00.276633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.276569    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="config-reloader" containerID="cri-o://9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317" gracePeriod=120
Oct 30 21:13:00.276879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.276371    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="alertmanager" containerID="cri-o://9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a" gracePeriod=120
Oct 30 21:13:00.276879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.276372    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy-metric" containerID="cri-o://28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395" gracePeriod=120
Oct 30 21:13:00.676845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676807    5786 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2" exitCode=0
Oct 30 21:13:00.676845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676831    5786 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395" exitCode=0
Oct 30 21:13:00.676845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676838    5786 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d" exitCode=0
Oct 30 21:13:00.676845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676844    5786 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317" exitCode=0
Oct 30 21:13:00.676845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676851    5786 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a" exitCode=0
Oct 30 21:13:00.677145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676874    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2"}
Oct 30 21:13:00.677145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676937    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395"}
Oct 30 21:13:00.677145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676964    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d"}
Oct 30 21:13:00.677145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.676988    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317"}
Oct 30 21:13:00.677145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:00.677011    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a"}
Oct 30 21:13:01.684649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:01.684619    5786 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8" exitCode=0
Oct 30 21:13:01.685032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:01.684669    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8"}
Oct 30 21:13:04.557272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:04.557242    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:13:04.557684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:04.557288    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:13:04.561905 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:04.561886    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:13:04.700186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:04.700157    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64c49d8448-l4drv"
Oct 30 21:13:04.841974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:04.841898    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-585d545cdf-xtfgw"]
Oct 30 21:13:06.492062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.492022    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-75579f5f75-plqsn" podUID="2007f2be-74e2-4db5-ba7e-c3e2596777d6" containerName="console" containerID="cri-o://5eee93f3fc491211ea3b4a3d1e354b76ff020611a823cf8166ae77242b24890e" gracePeriod=15
Oct 30 21:13:06.707271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.707237    5786 generic.go:356] "Generic (PLEG): container finished" podID="2007f2be-74e2-4db5-ba7e-c3e2596777d6" containerID="5eee93f3fc491211ea3b4a3d1e354b76ff020611a823cf8166ae77242b24890e" exitCode=0
Oct 30 21:13:06.707449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.707322    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75579f5f75-plqsn" event={"ID":"2007f2be-74e2-4db5-ba7e-c3e2596777d6","Type":"ContainerDied","Data":"5eee93f3fc491211ea3b4a3d1e354b76ff020611a823cf8166ae77242b24890e"}
Oct 30 21:13:06.782094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.782065    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:13:06.786966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.786945    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config\") pod \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") "
Oct 30 21:13:06.787041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.786983    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp\") pod \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") "
Oct 30 21:13:06.787041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787006    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5\" (UniqueName: \"kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5\") pod \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") "
Oct 30 21:13:06.787151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787040    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert\") pod \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") "
Oct 30 21:13:06.787151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787068    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert\") pod \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") "
Oct 30 21:13:06.787151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787106    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config\") pod \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") "
Oct 30 21:13:06.787317 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787150    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca\") pod \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\" (UID: \"2007f2be-74e2-4db5-ba7e-c3e2596777d6\") "
Oct 30 21:13:06.787403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787355    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp" pod "2007f2be-74e2-4db5-ba7e-c3e2596777d6" (UID: "2007f2be-74e2-4db5-ba7e-c3e2596777d6"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:13:06.787491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787469    5786 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2007f2be-74e2-4db5-ba7e-c3e2596777d6-tmp\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:06.787644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787619    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config" pod "2007f2be-74e2-4db5-ba7e-c3e2596777d6" (UID: "2007f2be-74e2-4db5-ba7e-c3e2596777d6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:13:06.787720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787706    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca" pod "2007f2be-74e2-4db5-ba7e-c3e2596777d6" (UID: "2007f2be-74e2-4db5-ba7e-c3e2596777d6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:13:06.787765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.787750    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert" pod "2007f2be-74e2-4db5-ba7e-c3e2596777d6" (UID: "2007f2be-74e2-4db5-ba7e-c3e2596777d6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:13:06.789720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.789699    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert" pod "2007f2be-74e2-4db5-ba7e-c3e2596777d6" (UID: "2007f2be-74e2-4db5-ba7e-c3e2596777d6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:13:06.789818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.789796    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5" pod "2007f2be-74e2-4db5-ba7e-c3e2596777d6" (UID: "2007f2be-74e2-4db5-ba7e-c3e2596777d6"). InnerVolumeSpecName "kube-api-access-jx5b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:13:06.789858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.789813    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config" pod "2007f2be-74e2-4db5-ba7e-c3e2596777d6" (UID: "2007f2be-74e2-4db5-ba7e-c3e2596777d6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:13:06.888705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.888671    5786 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-oauth-config\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:06.888705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.888698    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx5b5\" (UniqueName: \"kubernetes.io/projected/2007f2be-74e2-4db5-ba7e-c3e2596777d6-kube-api-access-jx5b5\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:06.888705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.888711    5786 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-serving-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:06.888978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.888720    5786 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-oauth-serving-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:06.888978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.888729    5786 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-console-config\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:06.888978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:06.888738    5786 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2007f2be-74e2-4db5-ba7e-c3e2596777d6-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:07.711954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:07.711928    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75579f5f75-plqsn"
Oct 30 21:13:07.712427 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:07.711923    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75579f5f75-plqsn" event={"ID":"2007f2be-74e2-4db5-ba7e-c3e2596777d6","Type":"ContainerDied","Data":"d60eaa159a472139d1e983915b0ee2aaaa5faff5a84b30404dbc2d8457c12066"}
Oct 30 21:13:07.712427 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:07.712065    5786 scope.go:122] "RemoveContainer" containerID="5eee93f3fc491211ea3b4a3d1e354b76ff020611a823cf8166ae77242b24890e"
Oct 30 21:13:07.872346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:07.872305    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75579f5f75-plqsn"]
Oct 30 21:13:07.921550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:07.921520    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75579f5f75-plqsn"]
Oct 30 21:13:08.357459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:08.357428    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2007f2be-74e2-4db5-ba7e-c3e2596777d6" path="/var/lib/kubelet/pods/2007f2be-74e2-4db5-ba7e-c3e2596777d6/volumes"
Oct 30 21:13:22.353285 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:13:22.353248    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:13:29.861718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:29.861655    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-585d545cdf-xtfgw" podUID="60e1e587-2825-4594-9c22-16fdd5ab0336" containerName="console" containerID="cri-o://11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b" gracePeriod=15
Oct 30 21:13:30.120732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.120678    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:13:30.190042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190012    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-oauth-serving-cert\") pod \"60e1e587-2825-4594-9c22-16fdd5ab0336\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") "
Oct 30 21:13:30.190157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190077    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-serving-cert\") pod \"60e1e587-2825-4594-9c22-16fdd5ab0336\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") "
Oct 30 21:13:30.190157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190102    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-trusted-ca-bundle\") pod \"60e1e587-2825-4594-9c22-16fdd5ab0336\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") "
Oct 30 21:13:30.190157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190131    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-service-ca\") pod \"60e1e587-2825-4594-9c22-16fdd5ab0336\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") "
Oct 30 21:13:30.190303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190168    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-oauth-config\") pod \"60e1e587-2825-4594-9c22-16fdd5ab0336\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") "
Oct 30 21:13:30.190303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190236    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-console-config\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-console-config\") pod \"60e1e587-2825-4594-9c22-16fdd5ab0336\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") "
Oct 30 21:13:30.190412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190305    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60e1e587-2825-4594-9c22-16fdd5ab0336-tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e1e587-2825-4594-9c22-16fdd5ab0336-tmp\") pod \"60e1e587-2825-4594-9c22-16fdd5ab0336\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") "
Oct 30 21:13:30.190412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190344    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60e1e587-2825-4594-9c22-16fdd5ab0336-kube-api-access-64gc9\" (UniqueName: \"kubernetes.io/projected/60e1e587-2825-4594-9c22-16fdd5ab0336-kube-api-access-64gc9\") pod \"60e1e587-2825-4594-9c22-16fdd5ab0336\" (UID: \"60e1e587-2825-4594-9c22-16fdd5ab0336\") "
Oct 30 21:13:30.190521 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190430    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-oauth-serving-cert" pod "60e1e587-2825-4594-9c22-16fdd5ab0336" (UID: "60e1e587-2825-4594-9c22-16fdd5ab0336"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:13:30.190601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190572    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e1e587-2825-4594-9c22-16fdd5ab0336-tmp" pod "60e1e587-2825-4594-9c22-16fdd5ab0336" (UID: "60e1e587-2825-4594-9c22-16fdd5ab0336"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:13:30.190764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190732    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-console-config" pod "60e1e587-2825-4594-9c22-16fdd5ab0336" (UID: "60e1e587-2825-4594-9c22-16fdd5ab0336"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:13:30.190764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190659    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-service-ca" pod "60e1e587-2825-4594-9c22-16fdd5ab0336" (UID: "60e1e587-2825-4594-9c22-16fdd5ab0336"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:13:30.190885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190803    5786 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e1e587-2825-4594-9c22-16fdd5ab0336-tmp\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:30.190885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190818    5786 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-oauth-serving-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:30.190885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.190840    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-trusted-ca-bundle" pod "60e1e587-2825-4594-9c22-16fdd5ab0336" (UID: "60e1e587-2825-4594-9c22-16fdd5ab0336"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:13:30.192625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.192592    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-oauth-config" pod "60e1e587-2825-4594-9c22-16fdd5ab0336" (UID: "60e1e587-2825-4594-9c22-16fdd5ab0336"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:13:30.192996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.192971    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-serving-cert" pod "60e1e587-2825-4594-9c22-16fdd5ab0336" (UID: "60e1e587-2825-4594-9c22-16fdd5ab0336"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:13:30.192996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.192986    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e1e587-2825-4594-9c22-16fdd5ab0336-kube-api-access-64gc9" pod "60e1e587-2825-4594-9c22-16fdd5ab0336" (UID: "60e1e587-2825-4594-9c22-16fdd5ab0336"). InnerVolumeSpecName "kube-api-access-64gc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:13:30.292065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.292032    5786 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-serving-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:30.292065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.292060    5786 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-trusted-ca-bundle\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:30.292287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.292078    5786 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:30.292287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.292087    5786 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e1e587-2825-4594-9c22-16fdd5ab0336-console-oauth-config\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:30.292287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.292095    5786 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e1e587-2825-4594-9c22-16fdd5ab0336-console-config\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:30.292287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.292112    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64gc9\" (UniqueName: \"kubernetes.io/projected/60e1e587-2825-4594-9c22-16fdd5ab0336-kube-api-access-64gc9\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:13:30.797875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.797837    5786 generic.go:356] "Generic (PLEG): container finished" podID="60e1e587-2825-4594-9c22-16fdd5ab0336" containerID="11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b" exitCode=0
Oct 30 21:13:30.798048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.797893    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-585d545cdf-xtfgw"
Oct 30 21:13:30.798048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.797910    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-585d545cdf-xtfgw" event={"ID":"60e1e587-2825-4594-9c22-16fdd5ab0336","Type":"ContainerDied","Data":"11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b"}
Oct 30 21:13:30.798048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.797961    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-585d545cdf-xtfgw" event={"ID":"60e1e587-2825-4594-9c22-16fdd5ab0336","Type":"ContainerDied","Data":"29430b653651cb5282ad669e6cfbe4b898d20f48ad92270b2c50f8e817c2ea98"}
Oct 30 21:13:30.798048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.797987    5786 scope.go:122] "RemoveContainer" containerID="11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b"
Oct 30 21:13:30.806122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.806102    5786 scope.go:122] "RemoveContainer" containerID="11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b"
Oct 30 21:13:30.806433 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:13:30.806414    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b\": container with ID starting with 11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b not found: ID does not exist" containerID="11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b"
Oct 30 21:13:30.806509 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.806443    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b"} err="failed to get container status \"11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b\": rpc error: code = NotFound desc = could not find container \"11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b\": container with ID starting with 11b5562134cfa9c2d8177861b7f4d0805c878f75d91d058f57f2214d93acf80b not found: ID does not exist"
Oct 30 21:13:30.959828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:30.959788    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-585d545cdf-xtfgw"]
Oct 30 21:13:31.009144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:31.009113    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-585d545cdf-xtfgw"]
Oct 30 21:13:32.356253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:13:32.356200    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60e1e587-2825-4594-9c22-16fdd5ab0336" path="/var/lib/kubelet/pods/60e1e587-2825-4594-9c22-16fdd5ab0336/volumes"
Oct 30 21:13:52.353621 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:13:52.353575    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:14:22.352699 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:14:22.352615    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:14:52.354199 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:14:52.354160    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:14:59.576964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.576941    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2f2767-292c-45d1-a573-fb2749f866fe/config-reloader/0.log"
Oct 30 21:14:59.578647 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.578628    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:59.625051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625015    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625058    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625081    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625121    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625143    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625162    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625180    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625244    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625295    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625346    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625376    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625403    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.625623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625442    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out\") pod \"ce2f2767-292c-45d1-a573-fb2749f866fe\" (UID: \"ce2f2767-292c-45d1-a573-fb2749f866fe\") "
Oct 30 21:14:59.626034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625632    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:14:59.626034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.625755    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-metrics-client-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.626142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.626092    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:14:59.628931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.628903    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:14:59.629066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.629003    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "kube-api-access-rgck7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:14:59.629134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.629058    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:14:59.629134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.629062    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:14:59.629374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.629350    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:14:59.629483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.629461    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:14:59.629549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.629495    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:14:59.629622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.629608    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:14:59.629876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.629851    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:14:59.633506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.633483    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:14:59.640995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.640966    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config" pod "ce2f2767-292c-45d1-a573-fb2749f866fe" (UID: "ce2f2767-292c-45d1-a573-fb2749f866fe"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:14:59.726673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726643    5786 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726669    5786 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-web-config\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726678    5786 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-config-out\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726688    5786 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-cluster-tls-config\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726697    5786 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726705    5786 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726713    5786 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-tls-assets\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726722    5786 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726732    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgck7\" (UniqueName: \"kubernetes.io/projected/ce2f2767-292c-45d1-a573-fb2749f866fe-kube-api-access-rgck7\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726741    5786 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2f2767-292c-45d1-a573-fb2749f866fe-secret-alertmanager-main-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726749    5786 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:14:59.726895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:14:59.726759    5786 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2f2767-292c-45d1-a573-fb2749f866fe-alertmanager-main-db\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:15:00.134101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.134072    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2f2767-292c-45d1-a573-fb2749f866fe/config-reloader/0.log"
Oct 30 21:15:00.134768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.134746    5786 generic.go:356] "Generic (PLEG): container finished" podID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerID="8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9" exitCode=137
Oct 30 21:15:00.134838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.134788    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9"}
Oct 30 21:15:00.134876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.134833    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.134876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.134851    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2f2767-292c-45d1-a573-fb2749f866fe","Type":"ContainerDied","Data":"2dea41af0a258a3b1a12868035a38ac48c13b4da0a66207a22e2e2058183566f"}
Oct 30 21:15:00.134944 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.134885    5786 scope.go:122] "RemoveContainer" containerID="f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2"
Oct 30 21:15:00.142577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.142558    5786 scope.go:122] "RemoveContainer" containerID="28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395"
Oct 30 21:15:00.150174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.150156    5786 scope.go:122] "RemoveContainer" containerID="984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d"
Oct 30 21:15:00.157027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.157010    5786 scope.go:122] "RemoveContainer" containerID="c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8"
Oct 30 21:15:00.163766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.163751    5786 scope.go:122] "RemoveContainer" containerID="9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317"
Oct 30 21:15:00.170807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.170792    5786 scope.go:122] "RemoveContainer" containerID="8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9"
Oct 30 21:15:00.177461 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.177443    5786 scope.go:122] "RemoveContainer" containerID="9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a"
Oct 30 21:15:00.184291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.184276    5786 scope.go:122] "RemoveContainer" containerID="d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2"
Oct 30 21:15:00.191488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.191469    5786 scope.go:122] "RemoveContainer" containerID="f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2"
Oct 30 21:15:00.191735 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:00.191716    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2\": container with ID starting with f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2 not found: ID does not exist" containerID="f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2"
Oct 30 21:15:00.191831 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.191745    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2"} err="failed to get container status \"f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2\": rpc error: code = NotFound desc = could not find container \"f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2\": container with ID starting with f07db74dcdf6fff734f390f5b756e89cda8329227efd8085fb781c4ea63145a2 not found: ID does not exist"
Oct 30 21:15:00.191831 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.191766    5786 scope.go:122] "RemoveContainer" containerID="28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395"
Oct 30 21:15:00.191990 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:00.191968    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395\": container with ID starting with 28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395 not found: ID does not exist" containerID="28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395"
Oct 30 21:15:00.192055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.192000    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395"} err="failed to get container status \"28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395\": rpc error: code = NotFound desc = could not find container \"28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395\": container with ID starting with 28998ecd30f093fbf0868fba712f60a204743f1a80a7c9e00d8dd0984d997395 not found: ID does not exist"
Oct 30 21:15:00.192055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.192023    5786 scope.go:122] "RemoveContainer" containerID="984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d"
Oct 30 21:15:00.192333 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:00.192315    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d\": container with ID starting with 984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d not found: ID does not exist" containerID="984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d"
Oct 30 21:15:00.192394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.192337    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d"} err="failed to get container status \"984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d\": rpc error: code = NotFound desc = could not find container \"984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d\": container with ID starting with 984d822467a90755c1ab19c191e636a097deb214f38286779f25e068e4790b0d not found: ID does not exist"
Oct 30 21:15:00.192394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.192351    5786 scope.go:122] "RemoveContainer" containerID="c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8"
Oct 30 21:15:00.192583 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:00.192564    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8\": container with ID starting with c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8 not found: ID does not exist" containerID="c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8"
Oct 30 21:15:00.192645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.192591    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8"} err="failed to get container status \"c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8\": rpc error: code = NotFound desc = could not find container \"c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8\": container with ID starting with c00656a849a26c37b707ee66d5065c249cb9a2b73a9e78d4e350b8469c2e27a8 not found: ID does not exist"
Oct 30 21:15:00.192645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.192611    5786 scope.go:122] "RemoveContainer" containerID="9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317"
Oct 30 21:15:00.192821 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:00.192806    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317\": container with ID starting with 9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317 not found: ID does not exist" containerID="9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317"
Oct 30 21:15:00.192873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.192824    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317"} err="failed to get container status \"9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317\": rpc error: code = NotFound desc = could not find container \"9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317\": container with ID starting with 9943b464599c7b7834c5f5dfe129acce56ff3b24f8f35e24a80fe20638539317 not found: ID does not exist"
Oct 30 21:15:00.192873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.192838    5786 scope.go:122] "RemoveContainer" containerID="8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9"
Oct 30 21:15:00.193003 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:00.192983    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9\": container with ID starting with 8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9 not found: ID does not exist" containerID="8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9"
Oct 30 21:15:00.193041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.193006    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9"} err="failed to get container status \"8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9\": rpc error: code = NotFound desc = could not find container \"8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9\": container with ID starting with 8ba9c3a14f51eb5c42c39b47e9f5f9004c34d05ef9d1649fe676149c864162a9 not found: ID does not exist"
Oct 30 21:15:00.193041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.193018    5786 scope.go:122] "RemoveContainer" containerID="9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a"
Oct 30 21:15:00.193188 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:00.193163    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a\": container with ID starting with 9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a not found: ID does not exist" containerID="9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a"
Oct 30 21:15:00.193246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.193191    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a"} err="failed to get container status \"9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a\": rpc error: code = NotFound desc = could not find container \"9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a\": container with ID starting with 9491c7e5f643a4eb349467cc64dc5502fb0777dd90e53ae99aa7f7ce45f7a83a not found: ID does not exist"
Oct 30 21:15:00.193246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.193223    5786 scope.go:122] "RemoveContainer" containerID="d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2"
Oct 30 21:15:00.193397 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:00.193379    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2\": container with ID starting with d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2 not found: ID does not exist" containerID="d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2"
Oct 30 21:15:00.193450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.193405    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2"} err="failed to get container status \"d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2\": rpc error: code = NotFound desc = could not find container \"d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2\": container with ID starting with d510558bd43f5e779b71451e05c312de2ff2768c32c7fb58b2832c2e7edbecb2 not found: ID does not exist"
Oct 30 21:15:00.299949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.299919    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:00.350052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350011    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:00.350547 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350530    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:00.350816 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350805    5786 cpu_manager.go:414] "containerMap: removing container" podUID="60e1e587-2825-4594-9c22-16fdd5ab0336" containerName="console"
Oct 30 21:15:00.350851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350818    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60e1e587-2825-4594-9c22-16fdd5ab0336" containerName="console"
Oct 30 21:15:00.350851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350831    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="config-reloader"
Oct 30 21:15:00.350851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350836    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="config-reloader"
Oct 30 21:15:00.350851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350842    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy"
Oct 30 21:15:00.350851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350847    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350857    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="init-config-reloader"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350862    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="init-config-reloader"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350868    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="config-reloader"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350872    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="config-reloader"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350884    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy-metric"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350889    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy-metric"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350895    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="alertmanager"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350899    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="alertmanager"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350906    5786 cpu_manager.go:414] "containerMap: removing container" podUID="2007f2be-74e2-4db5-ba7e-c3e2596777d6" containerName="console"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350910    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2007f2be-74e2-4db5-ba7e-c3e2596777d6" containerName="console"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350925    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy-web"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350931    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy-web"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350939    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="prom-label-proxy"
Oct 30 21:15:00.350994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.350943    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="prom-label-proxy"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351005    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="config-reloader"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351013    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy-metric"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351019    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="alertmanager"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351025    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="2007f2be-74e2-4db5-ba7e-c3e2596777d6" containerName="console"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351032    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy-web"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351037    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="prom-label-proxy"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351044    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="60e1e587-2825-4594-9c22-16fdd5ab0336" containerName="console"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351049    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="kube-rbac-proxy"
Oct 30 21:15:00.351390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.351144    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" containerName="config-reloader"
Oct 30 21:15:00.354143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.354128    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.359026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.359000    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce2f2767-292c-45d1-a573-fb2749f866fe" path="/var/lib/kubelet/pods/ce2f2767-292c-45d1-a573-fb2749f866fe/volumes"
Oct 30 21:15:00.406780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.406702    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:15:00.406780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.406707    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:15:00.406999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.406982    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:15:00.406999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.406990    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:15:00.407088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.407003    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:15:00.407088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.407050    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ljsb2\""
Oct 30 21:15:00.407240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.407196    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:15:00.407294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.407253    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:15:00.407326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.407299    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:15:00.432962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.432938    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c86fb99-95a6-4edc-98e0-19895d6aa48a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.432966    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c86fb99-95a6-4edc-98e0-19895d6aa48a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.432997    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433058    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433097    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433126    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c86fb99-95a6-4edc-98e0-19895d6aa48a-config-out\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433149    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c86fb99-95a6-4edc-98e0-19895d6aa48a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433173    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433249    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433297    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf27j\" (UniqueName: \"kubernetes.io/projected/0c86fb99-95a6-4edc-98e0-19895d6aa48a-kube-api-access-sf27j\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433328    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-web-config\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433354    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c86fb99-95a6-4edc-98e0-19895d6aa48a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.433428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.433376    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.456512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.456488    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:15:00.467384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.467286    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:00.534545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534509    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-web-config\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.534719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534551    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c86fb99-95a6-4edc-98e0-19895d6aa48a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.534719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534671    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.534719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534715    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c86fb99-95a6-4edc-98e0-19895d6aa48a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.534841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534735    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c86fb99-95a6-4edc-98e0-19895d6aa48a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.534841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534784    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.534841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534805    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.534841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534831    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.535044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534897    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c86fb99-95a6-4edc-98e0-19895d6aa48a-config-out\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.535044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534924    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c86fb99-95a6-4edc-98e0-19895d6aa48a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.535044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534937    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c86fb99-95a6-4edc-98e0-19895d6aa48a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.535044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.534985    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.535044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.535028    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.535348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.535070    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf27j\" (UniqueName: \"kubernetes.io/projected/0c86fb99-95a6-4edc-98e0-19895d6aa48a-kube-api-access-sf27j\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.535638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.535578    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c86fb99-95a6-4edc-98e0-19895d6aa48a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.535972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.535919    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c86fb99-95a6-4edc-98e0-19895d6aa48a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.538167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.538109    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-web-config\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.538293 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.538199    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.538358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.538340    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.538413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.538393    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.538588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.538561    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.538994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.538978    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.539084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.539065    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c86fb99-95a6-4edc-98e0-19895d6aa48a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.539293 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.539276    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c86fb99-95a6-4edc-98e0-19895d6aa48a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.539617 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.539595    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c86fb99-95a6-4edc-98e0-19895d6aa48a-config-out\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.592813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.592781    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf27j\" (UniqueName: \"kubernetes.io/projected/0c86fb99-95a6-4edc-98e0-19895d6aa48a-kube-api-access-sf27j\") pod \"alertmanager-main-0\" (UID: \"0c86fb99-95a6-4edc-98e0-19895d6aa48a\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:00.671616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:00.671589    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:01.098137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:01.098066    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:01.101588 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:15:01.101559    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c86fb99_95a6_4edc_98e0_19895d6aa48a.slice/crio-4afc4ad7bfb45a066ec81e3ed3ccd8b60932f07ec81ee39cabee430a43b790bd WatchSource:0}: Error finding container 4afc4ad7bfb45a066ec81e3ed3ccd8b60932f07ec81ee39cabee430a43b790bd: Status 404 returned error can't find the container with id 4afc4ad7bfb45a066ec81e3ed3ccd8b60932f07ec81ee39cabee430a43b790bd
Oct 30 21:15:01.140699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:01.140670    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c86fb99-95a6-4edc-98e0-19895d6aa48a","Type":"ContainerStarted","Data":"4afc4ad7bfb45a066ec81e3ed3ccd8b60932f07ec81ee39cabee430a43b790bd"}
Oct 30 21:15:02.145629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:02.145596    5786 generic.go:356] "Generic (PLEG): container finished" podID="0c86fb99-95a6-4edc-98e0-19895d6aa48a" containerID="fdd608c39dacf37c886135d1e589e725d24612ce11661a36e6244e5dcf88ac4c" exitCode=0
Oct 30 21:15:02.146006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:02.145648    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c86fb99-95a6-4edc-98e0-19895d6aa48a","Type":"ContainerDied","Data":"fdd608c39dacf37c886135d1e589e725d24612ce11661a36e6244e5dcf88ac4c"}
Oct 30 21:15:03.152991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:03.152952    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c86fb99-95a6-4edc-98e0-19895d6aa48a","Type":"ContainerStarted","Data":"5da3f535bf846699ed0998a9d477ea280465ced0b6cb0e77b1e4f71557f7b92d"}
Oct 30 21:15:03.153538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:03.153029    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c86fb99-95a6-4edc-98e0-19895d6aa48a","Type":"ContainerStarted","Data":"c765c0fc819bed18fd46ec7da57fcf9e6fdf29a2a80f7bd10bb8c4a22f66d917"}
Oct 30 21:15:03.153538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:03.153068    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c86fb99-95a6-4edc-98e0-19895d6aa48a","Type":"ContainerStarted","Data":"d2c97aa406f53b09a9fb1d13a122791203d133ccbbcae75ef023589a00f9dfb3"}
Oct 30 21:15:03.153538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:03.153112    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c86fb99-95a6-4edc-98e0-19895d6aa48a","Type":"ContainerStarted","Data":"6c624845b0361a8067b2408f3caf81a1ab2cb04041062c2556f445fc9a389ef3"}
Oct 30 21:15:03.153538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:03.153154    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c86fb99-95a6-4edc-98e0-19895d6aa48a","Type":"ContainerStarted","Data":"0cd3487364a1da4e39ba3bf5eed245da67298041beebd12071213d7d7efc1d8f"}
Oct 30 21:15:03.153538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:03.153192    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c86fb99-95a6-4edc-98e0-19895d6aa48a","Type":"ContainerStarted","Data":"5e38077bca5f4556b4c848499c135b85f77638ffee0bb65ed634a7461aebfe64"}
Oct 30 21:15:03.279097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:15:03.279046    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.267629745 podStartE2EDuration="3.279032679s" totalImagesPullingTime="0s" totalInitContainerRuntime="11.402934ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:03.278028527 +0000 UTC m=+159.149147658" watchObservedRunningTime="2026-10-30 21:15:03.279032679 +0000 UTC m=+159.150151829"
Oct 30 21:15:22.352005 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:22.351968    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:15:52.352854 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:15:52.352769    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:16:22.353196 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:16:22.353163    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:16:23.430792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.430755    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"]
Oct 30 21:16:23.434068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.434045    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.492173 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.492147    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:16:23.492379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.492223    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:16:23.533825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.533789    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.533959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.533850    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.534021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.533950    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.534021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.533991    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.534117 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.534054    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj554\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-kube-api-access-lj554\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.534166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.534148    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.534234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.534190    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.541756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.541732    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:16:23.541944 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.541752    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:16:23.542010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.541765    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:16:23.542010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.541812    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6b27n\""
Oct 30 21:16:23.542010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.541852    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:16:23.542755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.542730    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"]
Oct 30 21:16:23.634851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.634816    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.635048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.634867    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.635048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.634898    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.635048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.634937    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.635048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.634970    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.635048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.634991    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.635342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.635134    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj554\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-kube-api-access-lj554\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.635582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.635553    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.637463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.637441    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.637637 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.637610    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.637700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.637688    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.637836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.637819    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.694242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.694138    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj554\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-kube-api-access-lj554\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.695537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.695513    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-fxqnc\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:23.745850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:23.745813    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:24.169753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:24.169725    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"]
Oct 30 21:16:24.173409 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:16:24.173380    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacbfc40f_68e3_48e7_a1bb_8b5e3bce49c0.slice/crio-714eaff1abbb14c3b28ff0a7b45ea39a00d67b77e26f3867d9ff08a409f89a5e WatchSource:0}: Error finding container 714eaff1abbb14c3b28ff0a7b45ea39a00d67b77e26f3867d9ff08a409f89a5e: Status 404 returned error can't find the container with id 714eaff1abbb14c3b28ff0a7b45ea39a00d67b77e26f3867d9ff08a409f89a5e
Oct 30 21:16:24.436181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:24.436099    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc" event={"ID":"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0","Type":"ContainerStarted","Data":"714eaff1abbb14c3b28ff0a7b45ea39a00d67b77e26f3867d9ff08a409f89a5e"}
Oct 30 21:16:28.199174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:28.199143    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:16:28.199485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:28.199234    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:16:28.452963 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:28.452859    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc" event={"ID":"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0","Type":"ContainerStarted","Data":"67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5"}
Oct 30 21:16:28.453142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:28.453062    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:28.667977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:28.667884    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc" podStartSLOduration=1.644854161 podStartE2EDuration="5.667865519s" totalImagesPullingTime="4.023011358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:28.61785604 +0000 UTC m=+244.488975160" watchObservedRunningTime="2026-10-30 21:16:28.667865519 +0000 UTC m=+244.538984639"
Oct 30 21:16:29.458599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:29.458562    5786 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-fxqnc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:16:29.459054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:29.458664    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc" podUID="acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:16:32.457820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:16:32.457788    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:16:52.355158 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:16:52.355118    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:17:22.355312 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:17:22.355222    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:17:31.180504 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.180473    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"]
Oct 30 21:17:31.183852 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.183830    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.228521 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.228486    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.228690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.228555    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m62mg\" (UniqueName: \"kubernetes.io/projected/8222c43e-fb37-4521-94fd-a4698e0876e4-kube-api-access-m62mg\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.228690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.228644    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.228690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.228685    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.228854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.228702    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8222c43e-fb37-4521-94fd-a4698e0876e4-tmp\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.228854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.228829    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8222c43e-fb37-4521-94fd-a4698e0876e4-manifests\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.279728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.279694    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:17:31.279897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.279743    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:17:31.286034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.286004    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-d5dq4\""
Oct 30 21:17:31.286034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.286026    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:17:31.286192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.286100    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:17:31.286192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.286118    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:17:31.291195 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.291173    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"]
Oct 30 21:17:31.329566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.329533    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m62mg\" (UniqueName: \"kubernetes.io/projected/8222c43e-fb37-4521-94fd-a4698e0876e4-kube-api-access-m62mg\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.329740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.329576    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.329740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.329616    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.329740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.329647    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8222c43e-fb37-4521-94fd-a4698e0876e4-tmp\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.329740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.329679    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8222c43e-fb37-4521-94fd-a4698e0876e4-manifests\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.329740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.329722    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.330224 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.330177    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8222c43e-fb37-4521-94fd-a4698e0876e4-manifests\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.332389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.332361    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.332481 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.332419    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.332558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.332542    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8222c43e-fb37-4521-94fd-a4698e0876e4-tmp\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.332764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.332746    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8222c43e-fb37-4521-94fd-a4698e0876e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.388019 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.387981    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m62mg\" (UniqueName: \"kubernetes.io/projected/8222c43e-fb37-4521-94fd-a4698e0876e4-kube-api-access-m62mg\") pod \"opendatahub-operator-controller-manager-6c776cd74c-njc6b\" (UID: \"8222c43e-fb37-4521-94fd-a4698e0876e4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:31.500158 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:31.500071    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:32.015685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:32.015648    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"]
Oct 30 21:17:32.018957 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:17:32.018931    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8222c43e_fb37_4521_94fd_a4698e0876e4.slice/crio-3194675bf0267b5719c6d9ab9611e8c052f94e87c19318ffe4c9b944c75b6e29 WatchSource:0}: Error finding container 3194675bf0267b5719c6d9ab9611e8c052f94e87c19318ffe4c9b944c75b6e29: Status 404 returned error can't find the container with id 3194675bf0267b5719c6d9ab9611e8c052f94e87c19318ffe4c9b944c75b6e29
Oct 30 21:17:32.020850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:32.020832    5786 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:17:32.695134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:32.695104    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b" event={"ID":"8222c43e-fb37-4521-94fd-a4698e0876e4","Type":"ContainerStarted","Data":"3194675bf0267b5719c6d9ab9611e8c052f94e87c19318ffe4c9b944c75b6e29"}
Oct 30 21:17:35.711388 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:35.711288    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b" event={"ID":"8222c43e-fb37-4521-94fd-a4698e0876e4","Type":"ContainerStarted","Data":"add72d38aa5ced97a26bd70f58ad5a2742ba3d149c157763a4558fbac2bafc48"}
Oct 30 21:17:36.716835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:36.716802    5786 generic.go:356] "Generic (PLEG): container finished" podID="8222c43e-fb37-4521-94fd-a4698e0876e4" containerID="add72d38aa5ced97a26bd70f58ad5a2742ba3d149c157763a4558fbac2bafc48" exitCode=0
Oct 30 21:17:36.717477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:36.716861    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b" event={"ID":"8222c43e-fb37-4521-94fd-a4698e0876e4","Type":"ContainerDied","Data":"add72d38aa5ced97a26bd70f58ad5a2742ba3d149c157763a4558fbac2bafc48"}
Oct 30 21:17:37.723311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:37.723274    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b" event={"ID":"8222c43e-fb37-4521-94fd-a4698e0876e4","Type":"ContainerStarted","Data":"e91291e54357877ba92565dc288d60e488832f890d31f3b042ddaa9e3d1cc657"}
Oct 30 21:17:37.723758 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:37.723491    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:37.975120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:37.975065    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b" podStartSLOduration=3.149426493 podStartE2EDuration="7.975051008s" totalImagesPullingTime="4.271056132s" totalInitContainerRuntime="554.568383ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:37.925091657 +0000 UTC m=+313.796210778" watchObservedRunningTime="2026-10-30 21:17:37.975051008 +0000 UTC m=+313.846170128"
Oct 30 21:17:48.729987 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:17:48.729952    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6c776cd74c-njc6b"
Oct 30 21:17:52.354084 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:17:52.354043    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:18:21.022984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.022950    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55c8944d49-9c745"]
Oct 30 21:18:21.026368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.026345    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.096061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.096020    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05b9e5e9-fd34-46c1-9688-062a0c091b51-tls-certs\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.096267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.096081    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05b9e5e9-fd34-46c1-9688-062a0c091b51-tmp\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.096267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.096178    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgsvw\" (UniqueName: \"kubernetes.io/projected/05b9e5e9-fd34-46c1-9688-062a0c091b51-kube-api-access-hgsvw\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.130027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.129988    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55c8944d49-9c745"]
Oct 30 21:18:21.149647 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.149602    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:18:21.178984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.178950    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 21:18:21.179164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.179092    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fhwt5\""
Oct 30 21:18:21.197728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.197690    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05b9e5e9-fd34-46c1-9688-062a0c091b51-tls-certs\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.197902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.197764    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05b9e5e9-fd34-46c1-9688-062a0c091b51-tmp\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.197902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.197801    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgsvw\" (UniqueName: \"kubernetes.io/projected/05b9e5e9-fd34-46c1-9688-062a0c091b51-kube-api-access-hgsvw\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.200278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.200240    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05b9e5e9-fd34-46c1-9688-062a0c091b51-tmp\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.200546 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.200524    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05b9e5e9-fd34-46c1-9688-062a0c091b51-tls-certs\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.254715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.254675    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgsvw\" (UniqueName: \"kubernetes.io/projected/05b9e5e9-fd34-46c1-9688-062a0c091b51-kube-api-access-hgsvw\") pod \"kube-auth-proxy-55c8944d49-9c745\" (UID: \"05b9e5e9-fd34-46c1-9688-062a0c091b51\") " pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.338886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.338809    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745"
Oct 30 21:18:21.713069 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.713036    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55c8944d49-9c745"]
Oct 30 21:18:21.716836 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:18:21.716806    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b9e5e9_fd34_46c1_9688_062a0c091b51.slice/crio-25f49d22679b984bada44afc932e0c19a93c3cf5d4727a6a7ba1d94d6c568a45 WatchSource:0}: Error finding container 25f49d22679b984bada44afc932e0c19a93c3cf5d4727a6a7ba1d94d6c568a45: Status 404 returned error can't find the container with id 25f49d22679b984bada44afc932e0c19a93c3cf5d4727a6a7ba1d94d6c568a45
Oct 30 21:18:21.923604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:21.923569    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745" event={"ID":"05b9e5e9-fd34-46c1-9688-062a0c091b51","Type":"ContainerStarted","Data":"25f49d22679b984bada44afc932e0c19a93c3cf5d4727a6a7ba1d94d6c568a45"}
Oct 30 21:18:22.355148 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:18:22.355111    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:18:24.857375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:24.857337    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:18:24.944226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:24.944166    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745" event={"ID":"05b9e5e9-fd34-46c1-9688-062a0c091b51","Type":"ContainerStarted","Data":"17e9b72090651fe911f13a4c7e196ecb9dc3df98adde283ee57d01cc5c7259ab"}
Oct 30 21:18:25.060806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.060707    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55c8944d49-9c745" podStartSLOduration=1.973589678 podStartE2EDuration="5.060690781s" totalImagesPullingTime="3.087101103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:18:25.059550061 +0000 UTC m=+360.930669179" watchObservedRunningTime="2026-10-30 21:18:25.060690781 +0000 UTC m=+360.931809898"
Oct 30 21:18:25.194866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.194832    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fqmtx"]
Oct 30 21:18:25.198081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.198062    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-fqmtx"
Oct 30 21:18:25.258838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.258800    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:18:25.258838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.258825    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qvcbr\""
Oct 30 21:18:25.259339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.259323    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:18:25.279433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.279401    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fqmtx"]
Oct 30 21:18:25.336411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.336320    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcdxl\" (UniqueName: \"kubernetes.io/projected/dee8dce4-ec7e-4280-bb6e-1b257adfb64e-kube-api-access-gcdxl\") pod \"authorino-57f7f8659f-fqmtx\" (UID: \"dee8dce4-ec7e-4280-bb6e-1b257adfb64e\") " pod="kuadrant-system/authorino-57f7f8659f-fqmtx"
Oct 30 21:18:25.437651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.437600    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcdxl\" (UniqueName: \"kubernetes.io/projected/dee8dce4-ec7e-4280-bb6e-1b257adfb64e-kube-api-access-gcdxl\") pod \"authorino-57f7f8659f-fqmtx\" (UID: \"dee8dce4-ec7e-4280-bb6e-1b257adfb64e\") " pod="kuadrant-system/authorino-57f7f8659f-fqmtx"
Oct 30 21:18:25.500382 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.500341    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcdxl\" (UniqueName: \"kubernetes.io/projected/dee8dce4-ec7e-4280-bb6e-1b257adfb64e-kube-api-access-gcdxl\") pod \"authorino-57f7f8659f-fqmtx\" (UID: \"dee8dce4-ec7e-4280-bb6e-1b257adfb64e\") " pod="kuadrant-system/authorino-57f7f8659f-fqmtx"
Oct 30 21:18:25.521419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.521383    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-fqmtx"
Oct 30 21:18:25.889960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.889919    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fqmtx"]
Oct 30 21:18:25.894870 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:18:25.894840    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddee8dce4_ec7e_4280_bb6e_1b257adfb64e.slice/crio-e176ecb64bdd98df9c744f80d83a8774e1f1bf82a11c27b86512f8c5bd0eff91 WatchSource:0}: Error finding container e176ecb64bdd98df9c744f80d83a8774e1f1bf82a11c27b86512f8c5bd0eff91: Status 404 returned error can't find the container with id e176ecb64bdd98df9c744f80d83a8774e1f1bf82a11c27b86512f8c5bd0eff91
Oct 30 21:18:25.949957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:25.949918    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-fqmtx" event={"ID":"dee8dce4-ec7e-4280-bb6e-1b257adfb64e","Type":"ContainerStarted","Data":"e176ecb64bdd98df9c744f80d83a8774e1f1bf82a11c27b86512f8c5bd0eff91"}
Oct 30 21:18:27.836305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.836266    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"]
Oct 30 21:18:27.840886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.840862    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:27.942932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.942898    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-6jkjg\""
Oct 30 21:18:27.943155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.943039    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:18:27.943155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.943078    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 30 21:18:27.943324 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.943251    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 30 21:18:27.944336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.944313    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"]
Oct 30 21:18:27.961269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.961240    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e06518d1-5603-4766-ba64-8d71103f8025-cert\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:27.961390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.961294    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj4vf\" (UniqueName: \"kubernetes.io/projected/e06518d1-5603-4766-ba64-8d71103f8025-kube-api-access-pj4vf\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:27.961432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.961390    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e06518d1-5603-4766-ba64-8d71103f8025-metrics-certs\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:27.961486 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.961474    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e06518d1-5603-4766-ba64-8d71103f8025-manager-config\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:27.994628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.994597    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 30 21:18:27.994764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:27.994668    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 30 21:18:28.062604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.062567    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e06518d1-5603-4766-ba64-8d71103f8025-manager-config\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.062604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.062610    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e06518d1-5603-4766-ba64-8d71103f8025-cert\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.062821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.062652    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj4vf\" (UniqueName: \"kubernetes.io/projected/e06518d1-5603-4766-ba64-8d71103f8025-kube-api-access-pj4vf\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.062821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.062685    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e06518d1-5603-4766-ba64-8d71103f8025-metrics-certs\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.063260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.063229    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e06518d1-5603-4766-ba64-8d71103f8025-manager-config\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.065356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.065330    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e06518d1-5603-4766-ba64-8d71103f8025-metrics-certs\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.065356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.065351    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e06518d1-5603-4766-ba64-8d71103f8025-cert\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.119577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.119510    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj4vf\" (UniqueName: \"kubernetes.io/projected/e06518d1-5603-4766-ba64-8d71103f8025-kube-api-access-pj4vf\") pod \"jobset-controller-manager-78bd8688d6-29hbh\" (UID: \"e06518d1-5603-4766-ba64-8d71103f8025\") " pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.153416 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.153382    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:28.523396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.523365    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"]
Oct 30 21:18:28.526643 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:18:28.526609    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode06518d1_5603_4766_ba64_8d71103f8025.slice/crio-23c30259f0b864ef24ff014475dfacba1b4a0a88b1eec05637631a71725ccca6 WatchSource:0}: Error finding container 23c30259f0b864ef24ff014475dfacba1b4a0a88b1eec05637631a71725ccca6: Status 404 returned error can't find the container with id 23c30259f0b864ef24ff014475dfacba1b4a0a88b1eec05637631a71725ccca6
Oct 30 21:18:28.966980 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:28.966939    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh" event={"ID":"e06518d1-5603-4766-ba64-8d71103f8025","Type":"ContainerStarted","Data":"23c30259f0b864ef24ff014475dfacba1b4a0a88b1eec05637631a71725ccca6"}
Oct 30 21:18:30.979186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:30.979148    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-fqmtx" event={"ID":"dee8dce4-ec7e-4280-bb6e-1b257adfb64e","Type":"ContainerStarted","Data":"08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f"}
Oct 30 21:18:31.092045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:31.091977    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-fqmtx" podStartSLOduration=1.436172103 podStartE2EDuration="6.091955171s" totalImagesPullingTime="4.655783068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:18:31.091040721 +0000 UTC m=+366.962159850" watchObservedRunningTime="2026-10-30 21:18:31.091955171 +0000 UTC m=+366.963074289"
Oct 30 21:18:32.991452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:32.991417    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh" event={"ID":"e06518d1-5603-4766-ba64-8d71103f8025","Type":"ContainerStarted","Data":"cc410e7c0f517ba55bf6d94049f4681aa150dae58668ec8664b2ade82724fdfd"}
Oct 30 21:18:32.991842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:32.991557    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:33.107165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:33.107047    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh" podStartSLOduration=1.835472252 podStartE2EDuration="6.107030782s" totalImagesPullingTime="4.27155853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:18:33.106646344 +0000 UTC m=+368.977765487" watchObservedRunningTime="2026-10-30 21:18:33.107030782 +0000 UTC m=+368.978149901"
Oct 30 21:18:43.035555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:43.035518    5786 generic.go:356] "Generic (PLEG): container finished" podID="e06518d1-5603-4766-ba64-8d71103f8025" containerID="cc410e7c0f517ba55bf6d94049f4681aa150dae58668ec8664b2ade82724fdfd" exitCode=1
Oct 30 21:18:43.035959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:43.035571    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh" event={"ID":"e06518d1-5603-4766-ba64-8d71103f8025","Type":"ContainerDied","Data":"cc410e7c0f517ba55bf6d94049f4681aa150dae58668ec8664b2ade82724fdfd"}
Oct 30 21:18:43.035959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:43.035915    5786 scope.go:122] "RemoveContainer" containerID="cc410e7c0f517ba55bf6d94049f4681aa150dae58668ec8664b2ade82724fdfd"
Oct 30 21:18:44.041688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:44.041655    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh" event={"ID":"e06518d1-5603-4766-ba64-8d71103f8025","Type":"ContainerStarted","Data":"0940baa36b5d3924c91a744d9f31f194779f2f6d909db1511ce166b92f6f53a2"}
Oct 30 21:18:44.042119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:44.041890    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:18:50.037408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:50.037369    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fqmtx"]
Oct 30 21:18:50.037845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:50.037560    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-fqmtx" podUID="dee8dce4-ec7e-4280-bb6e-1b257adfb64e" containerName="authorino" containerID="cri-o://08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f" gracePeriod=30
Oct 30 21:18:50.279061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:50.279036    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-fqmtx"
Oct 30 21:18:50.358397 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:50.358326    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dee8dce4-ec7e-4280-bb6e-1b257adfb64e-kube-api-access-gcdxl\" (UniqueName: \"kubernetes.io/projected/dee8dce4-ec7e-4280-bb6e-1b257adfb64e-kube-api-access-gcdxl\") pod \"dee8dce4-ec7e-4280-bb6e-1b257adfb64e\" (UID: \"dee8dce4-ec7e-4280-bb6e-1b257adfb64e\") "
Oct 30 21:18:50.360567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:50.360541    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dee8dce4-ec7e-4280-bb6e-1b257adfb64e-kube-api-access-gcdxl" pod "dee8dce4-ec7e-4280-bb6e-1b257adfb64e" (UID: "dee8dce4-ec7e-4280-bb6e-1b257adfb64e"). InnerVolumeSpecName "kube-api-access-gcdxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:50.459843 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:50.459811    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcdxl\" (UniqueName: \"kubernetes.io/projected/dee8dce4-ec7e-4280-bb6e-1b257adfb64e-kube-api-access-gcdxl\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:18:51.072875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.072836    5786 generic.go:356] "Generic (PLEG): container finished" podID="dee8dce4-ec7e-4280-bb6e-1b257adfb64e" containerID="08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f" exitCode=0
Oct 30 21:18:51.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.072894    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-fqmtx"
Oct 30 21:18:51.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.072937    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-fqmtx" event={"ID":"dee8dce4-ec7e-4280-bb6e-1b257adfb64e","Type":"ContainerDied","Data":"08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f"}
Oct 30 21:18:51.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.072979    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-fqmtx" event={"ID":"dee8dce4-ec7e-4280-bb6e-1b257adfb64e","Type":"ContainerDied","Data":"e176ecb64bdd98df9c744f80d83a8774e1f1bf82a11c27b86512f8c5bd0eff91"}
Oct 30 21:18:51.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.072996    5786 scope.go:122] "RemoveContainer" containerID="08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f"
Oct 30 21:18:51.083945 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.083372    5786 scope.go:122] "RemoveContainer" containerID="08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f"
Oct 30 21:18:51.083945 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:18:51.083674    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f\": container with ID starting with 08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f not found: ID does not exist" containerID="08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f"
Oct 30 21:18:51.083945 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.083710    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f"} err="failed to get container status \"08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f\": rpc error: code = NotFound desc = could not find container \"08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f\": container with ID starting with 08730dcd5de1c288faf5f25f25108aa87f12156ef392327c90fdac6ef19b803f not found: ID does not exist"
Oct 30 21:18:51.233154 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.233117    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fqmtx"]
Oct 30 21:18:51.234897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:51.234876    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fqmtx"]
Oct 30 21:18:52.353971 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:18:52.353935    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:18:52.357801 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:52.357773    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dee8dce4-ec7e-4280-bb6e-1b257adfb64e" path="/var/lib/kubelet/pods/dee8dce4-ec7e-4280-bb6e-1b257adfb64e/volumes"
Oct 30 21:18:55.050991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:18:55.050962    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-78bd8688d6-29hbh"
Oct 30 21:19:22.353163 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:19:22.353123    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:19:33.081863 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.081823    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"]
Oct 30 21:19:33.082525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.082445    5786 cpu_manager.go:414] "containerMap: removing container" podUID="dee8dce4-ec7e-4280-bb6e-1b257adfb64e" containerName="authorino"
Oct 30 21:19:33.082525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.082465    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dee8dce4-ec7e-4280-bb6e-1b257adfb64e" containerName="authorino"
Oct 30 21:19:33.082652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.082572    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="dee8dce4-ec7e-4280-bb6e-1b257adfb64e" containerName="authorino"
Oct 30 21:19:33.084710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.084689    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.157803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.157771    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b85ec40-a08a-4220-b02a-9959b72251e1-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.157803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.157803    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9t96\" (UniqueName: \"kubernetes.io/projected/6b85ec40-a08a-4220-b02a-9959b72251e1-kube-api-access-w9t96\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.157998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.157833    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.157998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.157920    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.157998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.157949    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.157998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.157983    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.158114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.158099    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.186930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.186900    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"]
Oct 30 21:19:33.258540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.258508    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.258540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.258544    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.258796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.258571    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.258796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.258666    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.258796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.258693    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b85ec40-a08a-4220-b02a-9959b72251e1-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.258796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.258714    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9t96\" (UniqueName: \"kubernetes.io/projected/6b85ec40-a08a-4220-b02a-9959b72251e1-kube-api-access-w9t96\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.258991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.258857    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.259710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.259681    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.261027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.260999    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b85ec40-a08a-4220-b02a-9959b72251e1-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.261152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.261128    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.261234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.261163    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.261234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.261194    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.318061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.318025    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b85ec40-a08a-4220-b02a-9959b72251e1-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.318237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.318076    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9t96\" (UniqueName: \"kubernetes.io/projected/6b85ec40-a08a-4220-b02a-9959b72251e1-kube-api-access-w9t96\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lbxmv\" (UID: \"6b85ec40-a08a-4220-b02a-9959b72251e1\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.394838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.394760    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:33.814270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.814238    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"]
Oct 30 21:19:33.817398 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:19:33.817375    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b85ec40_a08a_4220_b02a_9959b72251e1.slice/crio-693145db14cb5d4aeb71afe3db96f2baeef67d3a1199943e31e3183665685788 WatchSource:0}: Error finding container 693145db14cb5d4aeb71afe3db96f2baeef67d3a1199943e31e3183665685788: Status 404 returned error can't find the container with id 693145db14cb5d4aeb71afe3db96f2baeef67d3a1199943e31e3183665685788
Oct 30 21:19:33.819416 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.819384    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:19:33.819478 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:33.819457    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:19:34.256995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:34.256959    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv" event={"ID":"6b85ec40-a08a-4220-b02a-9959b72251e1","Type":"ContainerStarted","Data":"d5eac852094b549c563da29ea8c7102ab6cb47c52634aba451c37510001ad90d"}
Oct 30 21:19:34.257391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:34.257017    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv" event={"ID":"6b85ec40-a08a-4220-b02a-9959b72251e1","Type":"ContainerStarted","Data":"693145db14cb5d4aeb71afe3db96f2baeef67d3a1199943e31e3183665685788"}
Oct 30 21:19:34.257391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:34.257116    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:34.258811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:34.258789    5786 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-lbxmv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:19:34.258936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:34.258834    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv" podUID="6b85ec40-a08a-4220-b02a-9959b72251e1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:19:34.374966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:34.374916    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv" podStartSLOduration=1.374902198 podStartE2EDuration="1.374902198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:19:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:19:34.373619034 +0000 UTC m=+430.244738153" watchObservedRunningTime="2026-10-30 21:19:34.374902198 +0000 UTC m=+430.246021317"
Oct 30 21:19:35.262487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:35.262453    5786 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-lbxmv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:19:35.262891 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:35.262515    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv" podUID="6b85ec40-a08a-4220-b02a-9959b72251e1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:19:38.263423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.263387    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lbxmv"
Oct 30 21:19:38.481584 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.481551    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"]
Oct 30 21:19:38.481804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.481762    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc" podUID="acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" containerName="discovery" containerID="cri-o://67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5" gracePeriod=30
Oct 30 21:19:38.719330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.719306    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:19:38.809092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.809061    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-local-certs\") pod \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") "
Oct 30 21:19:38.809092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.809105    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-cacerts\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-cacerts\") pod \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") "
Oct 30 21:19:38.809374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.809139    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-kubeconfig\") pod \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") "
Oct 30 21:19:38.809374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.809180    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-token\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-token\") pod \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") "
Oct 30 21:19:38.809374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.809253    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-kube-api-access-lj554\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-kube-api-access-lj554\") pod \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") "
Oct 30 21:19:38.809374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.809295    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-dns-cert\") pod \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") "
Oct 30 21:19:38.809374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.809348    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-ca-configmap\") pod \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\" (UID: \"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0\") "
Oct 30 21:19:38.810487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.810139    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-ca-configmap" pod "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" (UID: "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:19:38.811925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.811893    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-kube-api-access-lj554" pod "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" (UID: "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0"). InnerVolumeSpecName "kube-api-access-lj554". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:19:38.812044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.811994    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-local-certs" pod "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" (UID: "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:19:38.812101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.812040    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-kubeconfig" pod "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" (UID: "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:19:38.812101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.812083    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-dns-cert" pod "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" (UID: "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:19:38.812199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.812098    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-cacerts" pod "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" (UID: "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:19:38.812396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.812376    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-token" pod "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" (UID: "acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:19:38.911134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.911101    5786 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-local-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:19:38.911134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.911131    5786 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-cacerts\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:19:38.911134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.911142    5786 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-kubeconfig\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:19:38.911463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.911150    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:19:38.911463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.911159    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj554\" (UniqueName: \"kubernetes.io/projected/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-kube-api-access-lj554\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:19:38.911463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.911168    5786 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-dns-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:19:38.911463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:38.911177    5786 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0-istio-csr-ca-configmap\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:19:39.141324 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.141245    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-7dx2p"]
Oct 30 21:19:39.141592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.141580    5786 cpu_manager.go:414] "containerMap: removing container" podUID="acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" containerName="discovery"
Oct 30 21:19:39.141640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.141593    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" containerName="discovery"
Oct 30 21:19:39.141689 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.141679    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" containerName="discovery"
Oct 30 21:19:39.144100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.144082    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.195869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.195843    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 21:19:39.195869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.195844    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qfctw\""
Oct 30 21:19:39.196065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.195889    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 21:19:39.196065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.196033    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:19:39.196065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.196049    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:19:39.213542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.213515    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98ff4001-c3d7-40d3-88a8-d0d801917b7e-cert\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.213675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.213546    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr8pk\" (UniqueName: \"kubernetes.io/projected/98ff4001-c3d7-40d3-88a8-d0d801917b7e-kube-api-access-wr8pk\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.213675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.213568    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98ff4001-c3d7-40d3-88a8-d0d801917b7e-metrics-certs\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.246067 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.246046    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-7dx2p"]
Oct 30 21:19:39.281835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.281801    5786 generic.go:356] "Generic (PLEG): container finished" podID="acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" containerID="67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5" exitCode=0
Oct 30 21:19:39.282295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.281853    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"
Oct 30 21:19:39.282295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.281885    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc" event={"ID":"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0","Type":"ContainerDied","Data":"67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5"}
Oct 30 21:19:39.282295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.281932    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc" event={"ID":"acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0","Type":"ContainerDied","Data":"714eaff1abbb14c3b28ff0a7b45ea39a00d67b77e26f3867d9ff08a409f89a5e"}
Oct 30 21:19:39.282295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.281956    5786 scope.go:122] "RemoveContainer" containerID="67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5"
Oct 30 21:19:39.290904 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.290889    5786 scope.go:122] "RemoveContainer" containerID="67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5"
Oct 30 21:19:39.291157 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:19:39.291138    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5\": container with ID starting with 67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5 not found: ID does not exist" containerID="67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5"
Oct 30 21:19:39.291245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.291166    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5"} err="failed to get container status \"67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5\": rpc error: code = NotFound desc = could not find container \"67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5\": container with ID starting with 67dfc69a9f5e0cfdf14ee44a6c32968eb9705153207fd2435b238a0ece73baf5 not found: ID does not exist"
Oct 30 21:19:39.314065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.314045    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98ff4001-c3d7-40d3-88a8-d0d801917b7e-cert\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.314123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.314076    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr8pk\" (UniqueName: \"kubernetes.io/projected/98ff4001-c3d7-40d3-88a8-d0d801917b7e-kube-api-access-wr8pk\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.314123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.314100    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98ff4001-c3d7-40d3-88a8-d0d801917b7e-metrics-certs\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.316468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.316449    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98ff4001-c3d7-40d3-88a8-d0d801917b7e-cert\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.316574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.316550    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98ff4001-c3d7-40d3-88a8-d0d801917b7e-metrics-certs\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.370591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.370562    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr8pk\" (UniqueName: \"kubernetes.io/projected/98ff4001-c3d7-40d3-88a8-d0d801917b7e-kube-api-access-wr8pk\") pod \"odh-model-controller-769875db56-7dx2p\" (UID: \"98ff4001-c3d7-40d3-88a8-d0d801917b7e\") " pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.442671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.442645    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"]
Oct 30 21:19:39.446280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.446261    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-fxqnc"]
Oct 30 21:19:39.455563 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.455547    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:39.823011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:39.822943    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-7dx2p"]
Oct 30 21:19:39.826270 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:19:39.826239    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ff4001_c3d7_40d3_88a8_d0d801917b7e.slice/crio-e03fef8ebeaa09ec45a0749d584d3cc7abff73ab71b94b8905b9e5724d5eff08 WatchSource:0}: Error finding container e03fef8ebeaa09ec45a0749d584d3cc7abff73ab71b94b8905b9e5724d5eff08: Status 404 returned error can't find the container with id e03fef8ebeaa09ec45a0749d584d3cc7abff73ab71b94b8905b9e5724d5eff08
Oct 30 21:19:40.288845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:40.288802    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-7dx2p" event={"ID":"98ff4001-c3d7-40d3-88a8-d0d801917b7e","Type":"ContainerStarted","Data":"e03fef8ebeaa09ec45a0749d584d3cc7abff73ab71b94b8905b9e5724d5eff08"}
Oct 30 21:19:40.357781 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:40.357749    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0" path="/var/lib/kubelet/pods/acbfc40f-68e3-48e7-a1bb-8b5e3bce49c0/volumes"
Oct 30 21:19:43.305039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:43.304947    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-7dx2p" event={"ID":"98ff4001-c3d7-40d3-88a8-d0d801917b7e","Type":"ContainerStarted","Data":"a2f3e7a24e947d9bbc5357f0cec7d69a10de48e26f7a3648b45266a83e50e175"}
Oct 30 21:19:43.305418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:43.305082    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:19:43.419353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:43.419296    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-7dx2p" podStartSLOduration=1.283818157 podStartE2EDuration="4.419282484s" totalImagesPullingTime="3.135464327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:19:43.418344514 +0000 UTC m=+439.289463645" watchObservedRunningTime="2026-10-30 21:19:43.419282484 +0000 UTC m=+439.290401602"
Oct 30 21:19:52.353883 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:19:52.353851    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:19:53.347832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:53.347759    5786 generic.go:356] "Generic (PLEG): container finished" podID="98ff4001-c3d7-40d3-88a8-d0d801917b7e" containerID="a2f3e7a24e947d9bbc5357f0cec7d69a10de48e26f7a3648b45266a83e50e175" exitCode=1
Oct 30 21:19:53.347960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:53.347833    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-7dx2p" event={"ID":"98ff4001-c3d7-40d3-88a8-d0d801917b7e","Type":"ContainerDied","Data":"a2f3e7a24e947d9bbc5357f0cec7d69a10de48e26f7a3648b45266a83e50e175"}
Oct 30 21:19:53.348240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:53.348191    5786 scope.go:122] "RemoveContainer" containerID="a2f3e7a24e947d9bbc5357f0cec7d69a10de48e26f7a3648b45266a83e50e175"
Oct 30 21:19:54.357334 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:54.357303    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-7dx2p" event={"ID":"98ff4001-c3d7-40d3-88a8-d0d801917b7e","Type":"ContainerStarted","Data":"0304e1e80a341ce981bacb4b1709a2a5cd15f2f31fa97bf3dab23e456b6dddcc"}
Oct 30 21:19:54.357718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:19:54.357547    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:20:05.361341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:05.361266    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-7dx2p"
Oct 30 21:20:22.352735 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:20:22.352696    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:20:36.239648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.239614    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d745cf677-kqpx4"]
Oct 30 21:20:36.241765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.241748    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d745cf677-kqpx4"
Oct 30 21:20:36.288296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.288270    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kqc5\" (UniqueName: \"kubernetes.io/projected/14f76fb7-ce8f-4e67-a8a3-139e52f99882-kube-api-access-2kqc5\") pod \"authorino-6d745cf677-kqpx4\" (UID: \"14f76fb7-ce8f-4e67-a8a3-139e52f99882\") " pod="kuadrant-system/authorino-6d745cf677-kqpx4"
Oct 30 21:20:36.293760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.293746    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qvcbr\""
Oct 30 21:20:36.342802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.342780    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:20:36.342893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.342841    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:20:36.343925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.343909    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d745cf677-kqpx4"]
Oct 30 21:20:36.388845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.388821    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kqc5\" (UniqueName: \"kubernetes.io/projected/14f76fb7-ce8f-4e67-a8a3-139e52f99882-kube-api-access-2kqc5\") pod \"authorino-6d745cf677-kqpx4\" (UID: \"14f76fb7-ce8f-4e67-a8a3-139e52f99882\") " pod="kuadrant-system/authorino-6d745cf677-kqpx4"
Oct 30 21:20:36.445531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.445508    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kqc5\" (UniqueName: \"kubernetes.io/projected/14f76fb7-ce8f-4e67-a8a3-139e52f99882-kube-api-access-2kqc5\") pod \"authorino-6d745cf677-kqpx4\" (UID: \"14f76fb7-ce8f-4e67-a8a3-139e52f99882\") " pod="kuadrant-system/authorino-6d745cf677-kqpx4"
Oct 30 21:20:36.562041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.561975    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d745cf677-kqpx4"
Oct 30 21:20:36.690319 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.690292    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d745cf677-kqpx4"]
Oct 30 21:20:36.716845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.716820    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-889766f9f-v75mb"]
Oct 30 21:20:36.720277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.720260    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:20:36.771615 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.771595    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 21:20:36.776260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.776196    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-889766f9f-v75mb"]
Oct 30 21:20:36.791095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.791075    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbqx5\" (UniqueName: \"kubernetes.io/projected/f1480673-81c7-4737-be48-43ec7dc0ac0c-kube-api-access-jbqx5\") pod \"authorino-889766f9f-v75mb\" (UID: \"f1480673-81c7-4737-be48-43ec7dc0ac0c\") " pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:20:36.791192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.791132    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1480673-81c7-4737-be48-43ec7dc0ac0c-tls-cert\") pod \"authorino-889766f9f-v75mb\" (UID: \"f1480673-81c7-4737-be48-43ec7dc0ac0c\") " pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:20:36.892321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.892274    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1480673-81c7-4737-be48-43ec7dc0ac0c-tls-cert\") pod \"authorino-889766f9f-v75mb\" (UID: \"f1480673-81c7-4737-be48-43ec7dc0ac0c\") " pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:20:36.892418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.892332    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqx5\" (UniqueName: \"kubernetes.io/projected/f1480673-81c7-4737-be48-43ec7dc0ac0c-kube-api-access-jbqx5\") pod \"authorino-889766f9f-v75mb\" (UID: \"f1480673-81c7-4737-be48-43ec7dc0ac0c\") " pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:20:36.894697 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.894672    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1480673-81c7-4737-be48-43ec7dc0ac0c-tls-cert\") pod \"authorino-889766f9f-v75mb\" (UID: \"f1480673-81c7-4737-be48-43ec7dc0ac0c\") " pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:20:36.928746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.928727    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d745cf677-kqpx4"]
Oct 30 21:20:36.932094 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:20:36.932068    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14f76fb7_ce8f_4e67_a8a3_139e52f99882.slice/crio-f2214f17d6eb153c4999f68ecf59fe7d3954da3e8f10e447cc513c3cb26a6ef8 WatchSource:0}: Error finding container f2214f17d6eb153c4999f68ecf59fe7d3954da3e8f10e447cc513c3cb26a6ef8: Status 404 returned error can't find the container with id f2214f17d6eb153c4999f68ecf59fe7d3954da3e8f10e447cc513c3cb26a6ef8
Oct 30 21:20:36.948561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:36.948541    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqx5\" (UniqueName: \"kubernetes.io/projected/f1480673-81c7-4737-be48-43ec7dc0ac0c-kube-api-access-jbqx5\") pod \"authorino-889766f9f-v75mb\" (UID: \"f1480673-81c7-4737-be48-43ec7dc0ac0c\") " pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:20:37.031634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:37.031614    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:20:37.398089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:37.398061    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-889766f9f-v75mb"]
Oct 30 21:20:37.401035 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:20:37.401009    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1480673_81c7_4737_be48_43ec7dc0ac0c.slice/crio-3f78511883b820f3b71161c3b8dc6057e81662a0a7403d34938baa9e8db2412e WatchSource:0}: Error finding container 3f78511883b820f3b71161c3b8dc6057e81662a0a7403d34938baa9e8db2412e: Status 404 returned error can't find the container with id 3f78511883b820f3b71161c3b8dc6057e81662a0a7403d34938baa9e8db2412e
Oct 30 21:20:37.533718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:37.533677    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d745cf677-kqpx4" event={"ID":"14f76fb7-ce8f-4e67-a8a3-139e52f99882","Type":"ContainerStarted","Data":"f2214f17d6eb153c4999f68ecf59fe7d3954da3e8f10e447cc513c3cb26a6ef8"}
Oct 30 21:20:37.534801 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:37.534779    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-889766f9f-v75mb" event={"ID":"f1480673-81c7-4737-be48-43ec7dc0ac0c","Type":"ContainerStarted","Data":"3f78511883b820f3b71161c3b8dc6057e81662a0a7403d34938baa9e8db2412e"}
Oct 30 21:20:38.547928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:38.547890    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d745cf677-kqpx4" event={"ID":"14f76fb7-ce8f-4e67-a8a3-139e52f99882","Type":"ContainerStarted","Data":"7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd"}
Oct 30 21:20:38.548414 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:38.548102    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6d745cf677-kqpx4" podUID="14f76fb7-ce8f-4e67-a8a3-139e52f99882" containerName="authorino" containerID="cri-o://7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd" gracePeriod=30
Oct 30 21:20:38.660526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:38.660479    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6d745cf677-kqpx4" podStartSLOduration=1.1336416790000001 podStartE2EDuration="2.660462401s" totalImagesPullingTime="1.526820722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:38.659270184 +0000 UTC m=+494.530389304" watchObservedRunningTime="2026-10-30 21:20:38.660462401 +0000 UTC m=+494.531581516"
Oct 30 21:20:38.883236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:38.883197    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d745cf677-kqpx4"
Oct 30 21:20:39.011045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.010965    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14f76fb7-ce8f-4e67-a8a3-139e52f99882-kube-api-access-2kqc5\" (UniqueName: \"kubernetes.io/projected/14f76fb7-ce8f-4e67-a8a3-139e52f99882-kube-api-access-2kqc5\") pod \"14f76fb7-ce8f-4e67-a8a3-139e52f99882\" (UID: \"14f76fb7-ce8f-4e67-a8a3-139e52f99882\") "
Oct 30 21:20:39.013110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.013081    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f76fb7-ce8f-4e67-a8a3-139e52f99882-kube-api-access-2kqc5" pod "14f76fb7-ce8f-4e67-a8a3-139e52f99882" (UID: "14f76fb7-ce8f-4e67-a8a3-139e52f99882"). InnerVolumeSpecName "kube-api-access-2kqc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:20:39.112623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.112596    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kqc5\" (UniqueName: \"kubernetes.io/projected/14f76fb7-ce8f-4e67-a8a3-139e52f99882-kube-api-access-2kqc5\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:20:39.554276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.554232    5786 generic.go:356] "Generic (PLEG): container finished" podID="14f76fb7-ce8f-4e67-a8a3-139e52f99882" containerID="7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd" exitCode=0
Oct 30 21:20:39.554276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.554262    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d745cf677-kqpx4"
Oct 30 21:20:39.554813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.554308    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d745cf677-kqpx4" event={"ID":"14f76fb7-ce8f-4e67-a8a3-139e52f99882","Type":"ContainerDied","Data":"7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd"}
Oct 30 21:20:39.554813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.554350    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d745cf677-kqpx4" event={"ID":"14f76fb7-ce8f-4e67-a8a3-139e52f99882","Type":"ContainerDied","Data":"f2214f17d6eb153c4999f68ecf59fe7d3954da3e8f10e447cc513c3cb26a6ef8"}
Oct 30 21:20:39.554813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.554368    5786 scope.go:122] "RemoveContainer" containerID="7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd"
Oct 30 21:20:39.555975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.555953    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-889766f9f-v75mb" event={"ID":"f1480673-81c7-4737-be48-43ec7dc0ac0c","Type":"ContainerStarted","Data":"74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d"}
Oct 30 21:20:39.563673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.563504    5786 scope.go:122] "RemoveContainer" containerID="7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd"
Oct 30 21:20:39.565303 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:20:39.565050    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd\": container with ID starting with 7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd not found: ID does not exist" containerID="7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd"
Oct 30 21:20:39.565303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.565083    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd"} err="failed to get container status \"7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd\": rpc error: code = NotFound desc = could not find container \"7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd\": container with ID starting with 7c787ca5b423f9ba7115815a5290bbe9723b46b20c51b54d44949aa92eb548fd not found: ID does not exist"
Oct 30 21:20:39.722875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.722845    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d745cf677-kqpx4"]
Oct 30 21:20:39.723711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.723691    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d745cf677-kqpx4"]
Oct 30 21:20:39.833276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:39.833163    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-889766f9f-v75mb" podStartSLOduration=2.463005937 podStartE2EDuration="3.833144619s" totalImagesPullingTime="1.370138682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:39.832220565 +0000 UTC m=+495.703339681" watchObservedRunningTime="2026-10-30 21:20:39.833144619 +0000 UTC m=+495.704263739"
Oct 30 21:20:40.356936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:20:40.356903    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14f76fb7-ce8f-4e67-a8a3-139e52f99882" path="/var/lib/kubelet/pods/14f76fb7-ce8f-4e67-a8a3-139e52f99882/volumes"
Oct 30 21:20:52.352799 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:20:52.352760    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:21:22.353513 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:21:22.353408    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:21:52.352413 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:21:52.352330    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:22:07.364347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.364311    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"]
Oct 30 21:22:07.364715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.364669    5786 cpu_manager.go:414] "containerMap: removing container" podUID="14f76fb7-ce8f-4e67-a8a3-139e52f99882" containerName="authorino"
Oct 30 21:22:07.364715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.364679    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14f76fb7-ce8f-4e67-a8a3-139e52f99882" containerName="authorino"
Oct 30 21:22:07.364783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.364742    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="14f76fb7-ce8f-4e67-a8a3-139e52f99882" containerName="authorino"
Oct 30 21:22:07.367566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.367547    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.422047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.422012    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.422236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.422056    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfnbv\" (UniqueName: \"kubernetes.io/projected/6523484b-480f-4c20-b123-8013f0ef711e-kube-api-access-tfnbv\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.422236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.422129    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6523484b-480f-4c20-b123-8013f0ef711e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.422236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.422163    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.422353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.422311    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.422353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.422344    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.439037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.439012    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 21:22:07.469056 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.469033    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cpqws\""
Oct 30 21:22:07.469056 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.469034    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 21:22:07.469263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.469073    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 21:22:07.470991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.470970    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"]
Oct 30 21:22:07.522842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.522807    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.522842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.522843    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.523078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.522865    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.523078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.522902    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfnbv\" (UniqueName: \"kubernetes.io/projected/6523484b-480f-4c20-b123-8013f0ef711e-kube-api-access-tfnbv\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.523078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.522927    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6523484b-480f-4c20-b123-8013f0ef711e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.523078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.522947    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.523425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.523395    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.523541 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.523397    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.523541 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.523462    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.525343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.525323    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6523484b-480f-4c20-b123-8013f0ef711e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.525522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.525506    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6523484b-480f-4c20-b123-8013f0ef711e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.580311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.580284    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfnbv\" (UniqueName: \"kubernetes.io/projected/6523484b-480f-4c20-b123-8013f0ef711e-kube-api-access-tfnbv\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs\" (UID: \"6523484b-480f-4c20-b123-8013f0ef711e\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:07.679260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:07.679191    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:08.052424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:08.052345    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"]
Oct 30 21:22:08.055387 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:22:08.055358    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6523484b_480f_4c20_b123_8013f0ef711e.slice/crio-17408bc2e53a31dd1282986d5b9ba69be67e83cba66c0b66508d4bc1a2204047 WatchSource:0}: Error finding container 17408bc2e53a31dd1282986d5b9ba69be67e83cba66c0b66508d4bc1a2204047: Status 404 returned error can't find the container with id 17408bc2e53a31dd1282986d5b9ba69be67e83cba66c0b66508d4bc1a2204047
Oct 30 21:22:08.952032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:08.951990    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" event={"ID":"6523484b-480f-4c20-b123-8013f0ef711e","Type":"ContainerStarted","Data":"17408bc2e53a31dd1282986d5b9ba69be67e83cba66c0b66508d4bc1a2204047"}
Oct 30 21:22:09.962582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:09.962543    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" event={"ID":"6523484b-480f-4c20-b123-8013f0ef711e","Type":"ContainerStarted","Data":"54bf7cc6de751243f2b880879fc050fe6880b7932a28de81f2f406f749fd65ca"}
Oct 30 21:22:10.077050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:10.076988    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podStartSLOduration=1.5583622369999999 podStartE2EDuration="3.076963968s" totalImagesPullingTime="1.518601731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:10.076966512 +0000 UTC m=+585.948085632" watchObservedRunningTime="2026-10-30 21:22:10.076963968 +0000 UTC m=+585.948083089"
Oct 30 21:22:10.680308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:10.680270    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:14.696645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.696615    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"]
Oct 30 21:22:14.700368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.700348    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.792839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.792802    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.792839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.792840    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.793065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.792971    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.793065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.793005    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.793065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.793035    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll7zg\" (UniqueName: \"kubernetes.io/projected/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-kube-api-access-ll7zg\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.793167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.793125    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.801352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.801327    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 21:22:14.802565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.802541    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"]
Oct 30 21:22:14.893628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.893592    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.893800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.893644    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.893800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.893670    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.893800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.893754    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.893800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.893780    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.893953 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.893807    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll7zg\" (UniqueName: \"kubernetes.io/projected/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-kube-api-access-ll7zg\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.894097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.894078    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.894136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.894108    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.894172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.894155    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.896133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.896112    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.896517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.896497    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:14.956271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:14.956180    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll7zg\" (UniqueName: \"kubernetes.io/projected/086aa9d4-d49b-4f59-a95f-2a6eb8a82928-kube-api-access-ll7zg\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-8687n\" (UID: \"086aa9d4-d49b-4f59-a95f-2a6eb8a82928\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:15.012790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:15.012751    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:15.400768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:15.398658    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"]
Oct 30 21:22:15.405353 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:22:15.405326    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086aa9d4_d49b_4f59_a95f_2a6eb8a82928.slice/crio-397f15211f1abc8a0fd1e759cef9cb6e35162758e99da107e5c76eaf4d36f150 WatchSource:0}: Error finding container 397f15211f1abc8a0fd1e759cef9cb6e35162758e99da107e5c76eaf4d36f150: Status 404 returned error can't find the container with id 397f15211f1abc8a0fd1e759cef9cb6e35162758e99da107e5c76eaf4d36f150
Oct 30 21:22:15.989654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:15.989621    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n" event={"ID":"086aa9d4-d49b-4f59-a95f-2a6eb8a82928","Type":"ContainerStarted","Data":"397f15211f1abc8a0fd1e759cef9cb6e35162758e99da107e5c76eaf4d36f150"}
Oct 30 21:22:16.995921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:16.995886    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n" event={"ID":"086aa9d4-d49b-4f59-a95f-2a6eb8a82928","Type":"ContainerStarted","Data":"1d0e882a9b2c162736317d97a878c374c88e2b8b54cc40374081c2e28fd6d969"}
Oct 30 21:22:17.013259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:17.013227    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:17.113137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:17.113066    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n" podStartSLOduration=2.533326068 podStartE2EDuration="3.113049119s" totalImagesPullingTime="579.723051ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:17.112463132 +0000 UTC m=+592.983582270" watchObservedRunningTime="2026-10-30 21:22:17.113049119 +0000 UTC m=+592.984168240"
Oct 30 21:22:17.680375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:17.680330    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:17.692647 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:17.692622    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:18.011478 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:18.011403    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:22:22.353074 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:22:22.353042    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:22:24.108910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.108866    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"]
Oct 30 21:22:24.112292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.112275    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.213485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.213453    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 21:22:24.214788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.214762    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"]
Oct 30 21:22:24.283330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.283272    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.283523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.283357    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpd9q\" (UniqueName: \"kubernetes.io/projected/b5613d2c-edac-4610-9c85-03a98b6a8493-kube-api-access-mpd9q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.283523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.283429    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.283523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.283462    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.283523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.283494    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.283654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.283551    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5613d2c-edac-4610-9c85-03a98b6a8493-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.384474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.384392    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpd9q\" (UniqueName: \"kubernetes.io/projected/b5613d2c-edac-4610-9c85-03a98b6a8493-kube-api-access-mpd9q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.384474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.384445    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.384474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.384473    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.384702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.384508    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.384702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.384529    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5613d2c-edac-4610-9c85-03a98b6a8493-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.384702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.384574    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.384952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.384929    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.384990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.384941    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.385029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.385006    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.387175 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.387147    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5613d2c-edac-4610-9c85-03a98b6a8493-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.435421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.435386    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 21:22:24.441408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.441379    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpd9q\" (UniqueName: \"kubernetes.io/projected/b5613d2c-edac-4610-9c85-03a98b6a8493-kube-api-access-mpd9q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.447678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.447646    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5613d2c-edac-4610-9c85-03a98b6a8493-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb\" (UID: \"b5613d2c-edac-4610-9c85-03a98b6a8493\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:24.725102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:24.725065    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:25.013092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:25.013011    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:25.025570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:25.025535    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:25.042465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:25.042430    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-8687n"
Oct 30 21:22:25.102084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:25.102055    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"]
Oct 30 21:22:25.105267 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:22:25.105238    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5613d2c_edac_4610_9c85_03a98b6a8493.slice/crio-874b8cb3c2cd478f5f62bcab1c79afe1dee0b3318c72dbba006322aaab65753a WatchSource:0}: Error finding container 874b8cb3c2cd478f5f62bcab1c79afe1dee0b3318c72dbba006322aaab65753a: Status 404 returned error can't find the container with id 874b8cb3c2cd478f5f62bcab1c79afe1dee0b3318c72dbba006322aaab65753a
Oct 30 21:22:26.038328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:26.038236    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb" event={"ID":"b5613d2c-edac-4610-9c85-03a98b6a8493","Type":"ContainerStarted","Data":"0d029123357a4969e041621b760dac22ea72f3bd6e638256f564d2ea676dd3f7"}
Oct 30 21:22:26.038328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:26.038294    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb" event={"ID":"b5613d2c-edac-4610-9c85-03a98b6a8493","Type":"ContainerStarted","Data":"874b8cb3c2cd478f5f62bcab1c79afe1dee0b3318c72dbba006322aaab65753a"}
Oct 30 21:22:26.157011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:26.156958    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb" podStartSLOduration=1.543627106 podStartE2EDuration="2.156942142s" totalImagesPullingTime="613.315036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:26.155527769 +0000 UTC m=+602.026646888" watchObservedRunningTime="2026-10-30 21:22:26.156942142 +0000 UTC m=+602.028061260"
Oct 30 21:22:26.725176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:26.725147    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:33.502841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.502811    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"]
Oct 30 21:22:33.505354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.505337    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.561255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.561227    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6bm8h\""
Oct 30 21:22:33.581928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.581887    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.582088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.581955    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.582088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.582049    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.582194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.582086    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t962m\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-kube-api-access-t962m\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.582194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.582165    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/598e64a0-0e8a-4fe0-b218-2945257f3f74-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.582344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.582328    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.582393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.582379    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.582505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.582485    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.582555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.582522    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.610026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.609992    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"]
Oct 30 21:22:33.683708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.683669    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.683708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.683713    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.683924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.683833    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.683924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.683886    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.683989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.683925    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.683989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.683954    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.684136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.684117    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.684242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.684132    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t962m\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-kube-api-access-t962m\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.684242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.684179    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.684242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.684195    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/598e64a0-0e8a-4fe0-b218-2945257f3f74-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.684401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.684247    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.684401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.684326    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.684401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.684372    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.684912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.684889    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/598e64a0-0e8a-4fe0-b218-2945257f3f74-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.686377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.686355    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.686474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.686401    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.741959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.741924    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.742242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.742198    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t962m\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-kube-api-access-t962m\") pod \"maas-default-gateway-openshift-default-6cc458d887-v56rq\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:33.818513 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:33.818406    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:34.246329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:34.246297    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"]
Oct 30 21:22:34.250242 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:22:34.250190    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598e64a0_0e8a_4fe0_b218_2945257f3f74.slice/crio-830a1f47045237f975c9cb0e968f1a479309f304cdbbf0954696c642aa787f73 WatchSource:0}: Error finding container 830a1f47045237f975c9cb0e968f1a479309f304cdbbf0954696c642aa787f73: Status 404 returned error can't find the container with id 830a1f47045237f975c9cb0e968f1a479309f304cdbbf0954696c642aa787f73
Oct 30 21:22:34.252233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:34.252194    5786 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:22:34.725792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:34.725757    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:34.738440 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:34.738409    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:35.082202 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:35.082122    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" event={"ID":"598e64a0-0e8a-4fe0-b218-2945257f3f74","Type":"ContainerStarted","Data":"830a1f47045237f975c9cb0e968f1a479309f304cdbbf0954696c642aa787f73"}
Oct 30 21:22:35.093556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:35.093518    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb"
Oct 30 21:22:38.125292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:38.125262    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:22:38.125559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:38.125333    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:22:38.125559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:38.125364    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:22:39.102753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:39.102703    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" event={"ID":"598e64a0-0e8a-4fe0-b218-2945257f3f74","Type":"ContainerStarted","Data":"84472d51931686c18b7144e6c8bd254e51c214ba57cd1ab3c815b6d59a11210a"}
Oct 30 21:22:39.239425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:39.239356    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" podStartSLOduration=2.367362751 podStartE2EDuration="6.23933628s" totalImagesPullingTime="3.871973529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:39.236975158 +0000 UTC m=+615.108094279" watchObservedRunningTime="2026-10-30 21:22:39.23933628 +0000 UTC m=+615.110455401"
Oct 30 21:22:39.819420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:39.819374    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:39.821055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:39.821031    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v56rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:22:39.821114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:39.821099    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:22:40.819486 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:40.819451    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v56rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:22:40.819884 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:40.819510    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:22:41.818968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:41.818928    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v56rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:22:41.819155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:41.818990    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:22:42.819645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:42.819602    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v56rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:22:42.820139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:42.819664    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:22:43.819525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:43.819486    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v56rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:22:43.819700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:43.819553    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:22:44.819417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:44.819381    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v56rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:22:44.819632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:44.819457    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:22:46.312634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:46.312594    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:46.313143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:46.312932    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:46.313805 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:46.313784    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:22:48.635744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.635699    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"]
Oct 30 21:22:48.639476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.639451    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.760720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.760689    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.760900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.760727    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.760900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.760746    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/773d8fc2-85de-47dd-81b5-6831f66f5594-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.760900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.760808    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwlj8\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-kube-api-access-kwlj8\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.761000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.760923    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.761000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.760956    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.761000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.760979    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.761090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.760996    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.761122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.761092    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.847008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.846973    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"]
Oct 30 21:22:48.861753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861726    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.861921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861760    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.861921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861781    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/773d8fc2-85de-47dd-81b5-6831f66f5594-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.861921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861802    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwlj8\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-kube-api-access-kwlj8\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.861921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861833    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.861921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861865    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.861921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861898    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.861921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861919    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.862325 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.861945    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.862325 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.862223    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.862325 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.862246    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.862630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.862593    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/773d8fc2-85de-47dd-81b5-6831f66f5594-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.862743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.862716    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.862791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.862758    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.864520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.864504    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.864723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.864700    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.918902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.918871    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.919279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.919257    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwlj8\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-kube-api-access-kwlj8\") pod \"maas-default-gateway-openshift-default-6cc458d887-zr2km\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:48.952039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:48.952007    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:49.380523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:49.380443    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"]
Oct 30 21:22:49.384450 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:22:49.384422    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod773d8fc2_85de_47dd_81b5_6831f66f5594.slice/crio-5cb5db246f0c7ecfce9be467b84bbabe212d96661eab8969ef5770bf7f942ea2 WatchSource:0}: Error finding container 5cb5db246f0c7ecfce9be467b84bbabe212d96661eab8969ef5770bf7f942ea2: Status 404 returned error can't find the container with id 5cb5db246f0c7ecfce9be467b84bbabe212d96661eab8969ef5770bf7f942ea2
Oct 30 21:22:49.387343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:49.387314    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:22:49.387425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:49.387376    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:22:49.387425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:49.387403    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:22:50.174230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:50.173459    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" event={"ID":"773d8fc2-85de-47dd-81b5-6831f66f5594","Type":"ContainerStarted","Data":"3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e"}
Oct 30 21:22:50.174230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:50.173573    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" event={"ID":"773d8fc2-85de-47dd-81b5-6831f66f5594","Type":"ContainerStarted","Data":"5cb5db246f0c7ecfce9be467b84bbabe212d96661eab8969ef5770bf7f942ea2"}
Oct 30 21:22:50.293266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:50.293190    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podStartSLOduration=2.293174895 podStartE2EDuration="2.293174895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:50.292473736 +0000 UTC m=+626.163592887" watchObservedRunningTime="2026-10-30 21:22:50.293174895 +0000 UTC m=+626.164294014"
Oct 30 21:22:50.963048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:50.963015    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:50.963432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:50.963408    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-zr2km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:22:50.963621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:50.963595    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 21:22:51.963316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:51.962629    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-zr2km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:22:51.963316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:51.962717    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 21:22:52.380429 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:22:52.380353    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:22:52.963233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:52.955577    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-zr2km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:22:52.963233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:52.955676    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 21:22:53.962234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:53.953808    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-zr2km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:22:53.962234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:53.953873    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 21:22:54.966663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:54.966624    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-zr2km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:22:54.967349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:54.967311    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 21:22:55.422588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.422554    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-889766f9f-v75mb"]
Oct 30 21:22:55.422808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.422766    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-889766f9f-v75mb" podUID="f1480673-81c7-4737-be48-43ec7dc0ac0c" containerName="authorino" containerID="cri-o://74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d" gracePeriod=30
Oct 30 21:22:55.676695 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.676633    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:22:55.734500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.734471    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1480673-81c7-4737-be48-43ec7dc0ac0c-tls-cert\" (UniqueName: \"kubernetes.io/secret/f1480673-81c7-4737-be48-43ec7dc0ac0c-tls-cert\") pod \"f1480673-81c7-4737-be48-43ec7dc0ac0c\" (UID: \"f1480673-81c7-4737-be48-43ec7dc0ac0c\") "
Oct 30 21:22:55.734633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.734532    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1480673-81c7-4737-be48-43ec7dc0ac0c-kube-api-access-jbqx5\" (UniqueName: \"kubernetes.io/projected/f1480673-81c7-4737-be48-43ec7dc0ac0c-kube-api-access-jbqx5\") pod \"f1480673-81c7-4737-be48-43ec7dc0ac0c\" (UID: \"f1480673-81c7-4737-be48-43ec7dc0ac0c\") "
Oct 30 21:22:55.736716 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.736683    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1480673-81c7-4737-be48-43ec7dc0ac0c-kube-api-access-jbqx5" pod "f1480673-81c7-4737-be48-43ec7dc0ac0c" (UID: "f1480673-81c7-4737-be48-43ec7dc0ac0c"). InnerVolumeSpecName "kube-api-access-jbqx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:55.744987 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.744964    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1480673-81c7-4737-be48-43ec7dc0ac0c-tls-cert" pod "f1480673-81c7-4737-be48-43ec7dc0ac0c" (UID: "f1480673-81c7-4737-be48-43ec7dc0ac0c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:22:55.835914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.835873    5786 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1480673-81c7-4737-be48-43ec7dc0ac0c-tls-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:22:55.835914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.835901    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbqx5\" (UniqueName: \"kubernetes.io/projected/f1480673-81c7-4737-be48-43ec7dc0ac0c-kube-api-access-jbqx5\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:22:55.952849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.952753    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-zr2km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:22:55.952849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:55.952805    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 21:22:56.207818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.207721    5786 generic.go:356] "Generic (PLEG): container finished" podID="f1480673-81c7-4737-be48-43ec7dc0ac0c" containerID="74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d" exitCode=0
Oct 30 21:22:56.207818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.207757    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-889766f9f-v75mb" event={"ID":"f1480673-81c7-4737-be48-43ec7dc0ac0c","Type":"ContainerDied","Data":"74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d"}
Oct 30 21:22:56.207818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.207772    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-889766f9f-v75mb"
Oct 30 21:22:56.207818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.207801    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-889766f9f-v75mb" event={"ID":"f1480673-81c7-4737-be48-43ec7dc0ac0c","Type":"ContainerDied","Data":"3f78511883b820f3b71161c3b8dc6057e81662a0a7403d34938baa9e8db2412e"}
Oct 30 21:22:56.207818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.207822    5786 scope.go:122] "RemoveContainer" containerID="74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d"
Oct 30 21:22:56.217311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.217294    5786 scope.go:122] "RemoveContainer" containerID="74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d"
Oct 30 21:22:56.217607 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:22:56.217566    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d\": container with ID starting with 74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d not found: ID does not exist" containerID="74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d"
Oct 30 21:22:56.217685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.217616    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d"} err="failed to get container status \"74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d\": rpc error: code = NotFound desc = could not find container \"74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d\": container with ID starting with 74ec4d483910140e336564f48dad7006c56e19e0c14f8fa64e36308ec52ef42d not found: ID does not exist"
Oct 30 21:22:56.373833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.373800    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-889766f9f-v75mb"]
Oct 30 21:22:56.379901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:56.379877    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-889766f9f-v75mb"]
Oct 30 21:22:57.544233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:57.543976    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:57.545028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:57.545004    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:57.546144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:57.546125    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 21:22:58.358786 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:22:58.358758    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1480673-81c7-4737-be48-43ec7dc0ac0c" path="/var/lib/kubelet/pods/f1480673-81c7-4737-be48-43ec7dc0ac0c/volumes"
Oct 30 21:23:04.023114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.023043    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"]
Oct 30 21:23:04.023517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.023466    5786 cpu_manager.go:414] "containerMap: removing container" podUID="f1480673-81c7-4737-be48-43ec7dc0ac0c" containerName="authorino"
Oct 30 21:23:04.023517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.023477    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1480673-81c7-4737-be48-43ec7dc0ac0c" containerName="authorino"
Oct 30 21:23:04.023587 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.023534    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1480673-81c7-4737-be48-43ec7dc0ac0c" containerName="authorino"
Oct 30 21:23:04.026517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.026491    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:04.079112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.079063    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 21:23:04.119515 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.119482    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5s2m\" (UniqueName: \"kubernetes.io/projected/65d3c06a-089a-4595-a2fe-0c4a769b49aa-kube-api-access-f5s2m\") pod \"payload-pre-processing-7b98bb678c-6hstp\" (UID: \"65d3c06a-089a-4595-a2fe-0c4a769b49aa\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:04.119710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.119655    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65d3c06a-089a-4595-a2fe-0c4a769b49aa-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-6hstp\" (UID: \"65d3c06a-089a-4595-a2fe-0c4a769b49aa\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:04.127837 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.127819    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xvgq7\""
Oct 30 21:23:04.129364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.129345    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"]
Oct 30 21:23:04.178824 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.178789    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-z25fv"]
Oct 30 21:23:04.182292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.182269    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:04.221176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.221132    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5s2m\" (UniqueName: \"kubernetes.io/projected/65d3c06a-089a-4595-a2fe-0c4a769b49aa-kube-api-access-f5s2m\") pod \"payload-pre-processing-7b98bb678c-6hstp\" (UID: \"65d3c06a-089a-4595-a2fe-0c4a769b49aa\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:04.221399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.221190    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f94dd074-f073-4f4d-81eb-72ba79a39d9a-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-z25fv\" (UID: \"f94dd074-f073-4f4d-81eb-72ba79a39d9a\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:04.221399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.221269    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwvjp\" (UniqueName: \"kubernetes.io/projected/f94dd074-f073-4f4d-81eb-72ba79a39d9a-kube-api-access-kwvjp\") pod \"payload-processing-55fbbbf9dc-z25fv\" (UID: \"f94dd074-f073-4f4d-81eb-72ba79a39d9a\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:04.221500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.221425    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65d3c06a-089a-4595-a2fe-0c4a769b49aa-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-6hstp\" (UID: \"65d3c06a-089a-4595-a2fe-0c4a769b49aa\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:04.222168 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.222142    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65d3c06a-089a-4595-a2fe-0c4a769b49aa-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-6hstp\" (UID: \"65d3c06a-089a-4595-a2fe-0c4a769b49aa\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:04.231010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.230986    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-z25fv"]
Oct 30 21:23:04.278097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.278023    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5s2m\" (UniqueName: \"kubernetes.io/projected/65d3c06a-089a-4595-a2fe-0c4a769b49aa-kube-api-access-f5s2m\") pod \"payload-pre-processing-7b98bb678c-6hstp\" (UID: \"65d3c06a-089a-4595-a2fe-0c4a769b49aa\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:04.322886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.322847    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwvjp\" (UniqueName: \"kubernetes.io/projected/f94dd074-f073-4f4d-81eb-72ba79a39d9a-kube-api-access-kwvjp\") pod \"payload-processing-55fbbbf9dc-z25fv\" (UID: \"f94dd074-f073-4f4d-81eb-72ba79a39d9a\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:04.323089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.322998    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f94dd074-f073-4f4d-81eb-72ba79a39d9a-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-z25fv\" (UID: \"f94dd074-f073-4f4d-81eb-72ba79a39d9a\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:04.323722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.323692    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f94dd074-f073-4f4d-81eb-72ba79a39d9a-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-z25fv\" (UID: \"f94dd074-f073-4f4d-81eb-72ba79a39d9a\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:04.338667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.338637    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:04.381331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.381299    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwvjp\" (UniqueName: \"kubernetes.io/projected/f94dd074-f073-4f4d-81eb-72ba79a39d9a-kube-api-access-kwvjp\") pod \"payload-processing-55fbbbf9dc-z25fv\" (UID: \"f94dd074-f073-4f4d-81eb-72ba79a39d9a\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:04.499929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.499891    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:04.714377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.714350    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"]
Oct 30 21:23:04.718682 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:23:04.718652    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d3c06a_089a_4595_a2fe_0c4a769b49aa.slice/crio-b431543a52bf1b195637465bd79249e1f5e4482ce1a37581f6c0864d95a9c3ea WatchSource:0}: Error finding container b431543a52bf1b195637465bd79249e1f5e4482ce1a37581f6c0864d95a9c3ea: Status 404 returned error can't find the container with id b431543a52bf1b195637465bd79249e1f5e4482ce1a37581f6c0864d95a9c3ea
Oct 30 21:23:04.877763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:04.877621    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-z25fv"]
Oct 30 21:23:04.880409 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:23:04.880377    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94dd074_f073_4f4d_81eb_72ba79a39d9a.slice/crio-fe902d9fd9a019abd9a437f38cc337c9d95af37f12d478e55c7513bf4ffa1d8e WatchSource:0}: Error finding container fe902d9fd9a019abd9a437f38cc337c9d95af37f12d478e55c7513bf4ffa1d8e: Status 404 returned error can't find the container with id fe902d9fd9a019abd9a437f38cc337c9d95af37f12d478e55c7513bf4ffa1d8e
Oct 30 21:23:05.253799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:05.253753    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv" event={"ID":"f94dd074-f073-4f4d-81eb-72ba79a39d9a","Type":"ContainerStarted","Data":"fe902d9fd9a019abd9a437f38cc337c9d95af37f12d478e55c7513bf4ffa1d8e"}
Oct 30 21:23:05.255401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:05.255364    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp" event={"ID":"65d3c06a-089a-4595-a2fe-0c4a769b49aa","Type":"ContainerStarted","Data":"b431543a52bf1b195637465bd79249e1f5e4482ce1a37581f6c0864d95a9c3ea"}
Oct 30 21:23:08.275359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:08.275274    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv" event={"ID":"f94dd074-f073-4f4d-81eb-72ba79a39d9a","Type":"ContainerStarted","Data":"bb909314b40d1d3e23d5ba8be6a82de646b2ca7d534edd7f751ed1eaeb7933ce"}
Oct 30 21:23:08.275778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:08.275460    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:08.276834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:08.276809    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp" event={"ID":"65d3c06a-089a-4595-a2fe-0c4a769b49aa","Type":"ContainerStarted","Data":"6d4ed00414f7364679248f4b75a2b73e009ee7e6dc4c1180bef7e070e4179fe6"}
Oct 30 21:23:08.276948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:08.276926    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:08.391051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:08.391005    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv" podStartSLOduration=1.254397194 podStartE2EDuration="4.390991154s" totalImagesPullingTime="3.13659396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:08.389382835 +0000 UTC m=+644.260501954" watchObservedRunningTime="2026-10-30 21:23:08.390991154 +0000 UTC m=+644.262110270"
Oct 30 21:23:08.503687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:08.503631    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp" podStartSLOduration=2.273690872 podStartE2EDuration="5.503612923s" totalImagesPullingTime="3.229922051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:08.503303979 +0000 UTC m=+644.374423098" watchObservedRunningTime="2026-10-30 21:23:08.503612923 +0000 UTC m=+644.374732044"
Oct 30 21:23:11.660086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.660051    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-76d5869bc7-svshz"]
Oct 30 21:23:11.664557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.664539    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:11.719246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.719221    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-76d5869bc7-svshz"]
Oct 30 21:23:11.766790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.766772    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-5c6fz\""
Oct 30 21:23:11.766790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.766773    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 21:23:11.766926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.766823    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 21:23:11.801893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.801869    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8gwm\" (UniqueName: \"kubernetes.io/projected/ec5111f7-df66-489c-8fe3-95c2a03aa914-kube-api-access-v8gwm\") pod \"ai-gateway-controller-76d5869bc7-svshz\" (UID: \"ec5111f7-df66-489c-8fe3-95c2a03aa914\") " pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:11.801984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.801922    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec5111f7-df66-489c-8fe3-95c2a03aa914-metrics-certs\") pod \"ai-gateway-controller-76d5869bc7-svshz\" (UID: \"ec5111f7-df66-489c-8fe3-95c2a03aa914\") " pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:11.902383 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.902360    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8gwm\" (UniqueName: \"kubernetes.io/projected/ec5111f7-df66-489c-8fe3-95c2a03aa914-kube-api-access-v8gwm\") pod \"ai-gateway-controller-76d5869bc7-svshz\" (UID: \"ec5111f7-df66-489c-8fe3-95c2a03aa914\") " pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:11.902468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.902413    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec5111f7-df66-489c-8fe3-95c2a03aa914-metrics-certs\") pod \"ai-gateway-controller-76d5869bc7-svshz\" (UID: \"ec5111f7-df66-489c-8fe3-95c2a03aa914\") " pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:11.904942 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.904921    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec5111f7-df66-489c-8fe3-95c2a03aa914-metrics-certs\") pod \"ai-gateway-controller-76d5869bc7-svshz\" (UID: \"ec5111f7-df66-489c-8fe3-95c2a03aa914\") " pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:11.959292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.959265    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8gwm\" (UniqueName: \"kubernetes.io/projected/ec5111f7-df66-489c-8fe3-95c2a03aa914-kube-api-access-v8gwm\") pod \"ai-gateway-controller-76d5869bc7-svshz\" (UID: \"ec5111f7-df66-489c-8fe3-95c2a03aa914\") " pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:11.975961 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:11.975942    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:12.341823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:12.341747    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-76d5869bc7-svshz"]
Oct 30 21:23:12.345711 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:23:12.345687    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec5111f7_df66_489c_8fe3_95c2a03aa914.slice/crio-fe1382fa2f0a5b05477ea9b654f1660e934e9324429804b83589a9f70662618d WatchSource:0}: Error finding container fe1382fa2f0a5b05477ea9b654f1660e934e9324429804b83589a9f70662618d: Status 404 returned error can't find the container with id fe1382fa2f0a5b05477ea9b654f1660e934e9324429804b83589a9f70662618d
Oct 30 21:23:13.298684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:13.298653    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz" event={"ID":"ec5111f7-df66-489c-8fe3-95c2a03aa914","Type":"ContainerStarted","Data":"fe1382fa2f0a5b05477ea9b654f1660e934e9324429804b83589a9f70662618d"}
Oct 30 21:23:15.309304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:15.309268    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz" event={"ID":"ec5111f7-df66-489c-8fe3-95c2a03aa914","Type":"ContainerStarted","Data":"7b50a0cd4617b4a97bddd22cac57b7b5b8ddedec0685ad514f89b24bb210b0d8"}
Oct 30 21:23:15.309717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:15.309412    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:15.424097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:15.424050    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz" podStartSLOduration=1.650777815 podStartE2EDuration="4.424036587s" totalImagesPullingTime="2.773258772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:15.423708112 +0000 UTC m=+651.294827231" watchObservedRunningTime="2026-10-30 21:23:15.424036587 +0000 UTC m=+651.295155705"
Oct 30 21:23:19.284469 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:19.284443    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-z25fv"
Oct 30 21:23:19.284936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:19.284510    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hstp"
Oct 30 21:23:22.354882 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:23:22.354846    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:23:26.316714 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:26.316682    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-76d5869bc7-svshz"
Oct 30 21:23:33.988259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:33.988220    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"]
Oct 30 21:23:33.990958 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:33.990940    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006237    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006388 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006272    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006388 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006294    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006436    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006540    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsk82\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-kube-api-access-vsk82\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006588    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006610    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b786b9-4afd-4f00-b1bc-4af69e49643a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006627    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.006709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.006683    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.102593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.102563    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"]
Oct 30 21:23:34.107915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.107887    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.107922    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.107948    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.107982    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108044    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsk82\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-kube-api-access-vsk82\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108099    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108126    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b786b9-4afd-4f00-b1bc-4af69e49643a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108151    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108183    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108434    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108475    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.108958 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108930    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b786b9-4afd-4f00-b1bc-4af69e49643a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.109110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.108959    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.109327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.109306    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.111033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.111003    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.111220 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.111189    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.165280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.165250    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.165459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.165438    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsk82\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-kube-api-access-vsk82\") pod \"maas-default-gateway-openshift-default-6cc458d887-clmqd\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.306079 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.305982    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:34.744612 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.744587    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"]
Oct 30 21:23:34.747158 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:23:34.747133    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b786b9_4afd_4f00_b1bc_4af69e49643a.slice/crio-78ce5cbd0ec54677d4657e5914202f47ac317c433659eabc38b502b421a4893b WatchSource:0}: Error finding container 78ce5cbd0ec54677d4657e5914202f47ac317c433659eabc38b502b421a4893b: Status 404 returned error can't find the container with id 78ce5cbd0ec54677d4657e5914202f47ac317c433659eabc38b502b421a4893b
Oct 30 21:23:34.749272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.749240    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:23:34.749358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.749307    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:23:34.749358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:34.749333    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:23:35.401518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:35.401487    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" event={"ID":"16b786b9-4afd-4f00-b1bc-4af69e49643a","Type":"ContainerStarted","Data":"a79a64b5de6c31af2aaf9ff9fabed5bb99ff12d9d7f342cadbb50464cc8d1472"}
Oct 30 21:23:35.401939 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:35.401539    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" event={"ID":"16b786b9-4afd-4f00-b1bc-4af69e49643a","Type":"ContainerStarted","Data":"78ce5cbd0ec54677d4657e5914202f47ac317c433659eabc38b502b421a4893b"}
Oct 30 21:23:36.307106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:36.307065    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:36.308789 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:36.308762    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-clmqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:23:36.308930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:36.308813    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:23:37.307068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:37.307028    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-clmqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:23:37.307499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:37.307087    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:23:38.306990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:38.306953    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-clmqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:23:38.307187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:38.307008    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:23:39.307035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:39.307003    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-clmqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:23:39.307200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:39.307059    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:23:40.306674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:40.306633    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-clmqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:23:40.306887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:40.306701    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:23:41.306760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:41.306708    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-clmqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:23:41.307161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:41.306794    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:23:42.953019 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:42.952980    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:42.953643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:42.953587    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:42.954377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:42.954356    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:23:43.120949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:23:43.120893    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" podStartSLOduration=10.120879802 podStartE2EDuration="10.120879802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:35.57629557 +0000 UTC m=+671.447414694" watchObservedRunningTime="2026-10-30 21:23:43.120879802 +0000 UTC m=+678.991998921"
Oct 30 21:23:52.353353 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:23:52.353317    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:24:03.818433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.818390    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"]
Oct 30 21:24:03.821836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.821817    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.873535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.873499    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-847c62-openshift-default-dockercfg-67hhh\""
Oct 30 21:24:03.924972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.924941    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"]
Oct 30 21:24:03.940808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.940782    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-credential-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.940957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.940821    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-podinfo\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.940957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.940839    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istiod-ca-cert\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.941032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.940971    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-envoy\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.941032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.941010    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-data\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.941099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.941051    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-token\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.941163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.941140    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.941301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.941282    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwzvl\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-kube-api-access-xwzvl\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:03.941359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:03.941344    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-certs\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042320    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwzvl\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-kube-api-access-xwzvl\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042353    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-certs\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042394    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-credential-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042433    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-podinfo\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042454    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istiod-ca-cert\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042487    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-envoy\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042519    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-data\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042562    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-token\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042614    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.042898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042770    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-certs\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.043005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.042945    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.043049 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.043003    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-credential-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.043119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.043089    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-data\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.043173 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.043131    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istiod-ca-cert\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.045022 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.044999    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-envoy\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.045393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.045369    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-podinfo\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.101494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.101440    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-token\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.101847 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.101827    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwzvl\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-kube-api-access-xwzvl\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.134558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.134536    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:04.426642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.426612    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"]
Oct 30 21:24:04.430329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.430305    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447325 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447302    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447374    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447400    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca508309-2ab8-420f-a9d1-f26963632e25-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447539 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447508    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4jkm\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-kube-api-access-s4jkm\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447539    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447563    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447602    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447722    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.447775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.447759    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca508309-2ab8-420f-a9d1-f26963632e25-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.481734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.481700    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-0a4eab-openshift-default-dockercfg-f7j2g\""
Oct 30 21:24:04.492619 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.492588    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"]
Oct 30 21:24:04.548896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.548854    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.548896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.548897    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca508309-2ab8-420f-a9d1-f26963632e25-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.548952    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.548999    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549022    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca508309-2ab8-420f-a9d1-f26963632e25-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549079    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4jkm\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-kube-api-access-s4jkm\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549115    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549150    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549176    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549481    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549502    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549617    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.549704 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.549653    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.550145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.550125    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca508309-2ab8-420f-a9d1-f26963632e25-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.551342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.551327    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.551558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.551543    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca508309-2ab8-420f-a9d1-f26963632e25-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.609389 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:04.609339    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a36ca42_23dc_46ef_a105_cf2d9ee227f0.slice/crio-db8a955e2bb64189e8c2d59b24037972c6c8d81c905d03f6007fa3ce2f8b488e WatchSource:0}: Error finding container db8a955e2bb64189e8c2d59b24037972c6c8d81c905d03f6007fa3ce2f8b488e: Status 404 returned error can't find the container with id db8a955e2bb64189e8c2d59b24037972c6c8d81c905d03f6007fa3ce2f8b488e
Oct 30 21:24:04.612316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.612162    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:04.612316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.612309    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:04.612482 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.612362    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:04.613126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.612870    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"]
Oct 30 21:24:04.613126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.613069    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.614876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.614855    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4jkm\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-kube-api-access-s4jkm\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:04.743669 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:04.743589    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:05.098310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.098241    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"]
Oct 30 21:24:05.103066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.103046    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.154897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.154861    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-38b832-openshift-default-dockercfg-q7nbg\""
Oct 30 21:24:05.156267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156240    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.156406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156326    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3760152e-c736-402c-89b3-b862b8fd1193-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.156406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156355    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt6tn\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-kube-api-access-zt6tn\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.156512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156407    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.156562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156527    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.156562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156557    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.156660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156575    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3760152e-c736-402c-89b3-b862b8fd1193-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.156660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156623    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.156756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.156657    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.161389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.161368    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"]
Oct 30 21:24:05.176764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.176704    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"]
Oct 30 21:24:05.179155 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:05.179129    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca508309_2ab8_420f_a9d1_f26963632e25.slice/crio-364eedf102698b2193bb41e4989495beaf53fb3da768d644a991ac1cbf43cb57 WatchSource:0}: Error finding container 364eedf102698b2193bb41e4989495beaf53fb3da768d644a991ac1cbf43cb57: Status 404 returned error can't find the container with id 364eedf102698b2193bb41e4989495beaf53fb3da768d644a991ac1cbf43cb57
Oct 30 21:24:05.181796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.181762    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:05.181864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.181848    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:05.181908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.181895    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:05.258390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258340    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.258568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258445    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3760152e-c736-402c-89b3-b862b8fd1193-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.258568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258483    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt6tn\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-kube-api-access-zt6tn\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.258568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258553    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.258725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258628    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.258725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258658    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.258725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258680    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3760152e-c736-402c-89b3-b862b8fd1193-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.258868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258731    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.258868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.258753    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.261420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.259328    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.261420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.260363    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3760152e-c736-402c-89b3-b862b8fd1193-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.261625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.261445    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.263323 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.261840    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.264852 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.264828    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.266810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.266739    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3760152e-c736-402c-89b3-b862b8fd1193-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.267633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.267603    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.318038 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.318006    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.318482 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.318464    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt6tn\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-kube-api-access-zt6tn\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.416593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.416564    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:05.544192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.544026    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" event={"ID":"2a36ca42-23dc-46ef-a105-cf2d9ee227f0","Type":"ContainerStarted","Data":"fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e"}
Oct 30 21:24:05.544192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.544092    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" event={"ID":"2a36ca42-23dc-46ef-a105-cf2d9ee227f0","Type":"ContainerStarted","Data":"db8a955e2bb64189e8c2d59b24037972c6c8d81c905d03f6007fa3ce2f8b488e"}
Oct 30 21:24:05.546093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.546068    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" event={"ID":"ca508309-2ab8-420f-a9d1-f26963632e25","Type":"ContainerStarted","Data":"3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3"}
Oct 30 21:24:05.546230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.546113    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" event={"ID":"ca508309-2ab8-420f-a9d1-f26963632e25","Type":"ContainerStarted","Data":"364eedf102698b2193bb41e4989495beaf53fb3da768d644a991ac1cbf43cb57"}
Oct 30 21:24:05.663318 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.663265    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" podStartSLOduration=2.66325008 podStartE2EDuration="2.66325008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:05.661410179 +0000 UTC m=+701.532529310" watchObservedRunningTime="2026-10-30 21:24:05.66325008 +0000 UTC m=+701.534369199"
Oct 30 21:24:05.743765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.743694    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:05.788244 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.779453    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podStartSLOduration=1.779386994 podStartE2EDuration="1.779386994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:05.777286402 +0000 UTC m=+701.648405522" watchObservedRunningTime="2026-10-30 21:24:05.779386994 +0000 UTC m=+701.650506114"
Oct 30 21:24:05.881232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.881193    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"]
Oct 30 21:24:05.883716 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:05.883695    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3760152e_c736_402c_89b3_b862b8fd1193.slice/crio-064a94e6973e7a68c99f19f5e4fe3196145baca559935838e86a3d21f91bdeba WatchSource:0}: Error finding container 064a94e6973e7a68c99f19f5e4fe3196145baca559935838e86a3d21f91bdeba: Status 404 returned error can't find the container with id 064a94e6973e7a68c99f19f5e4fe3196145baca559935838e86a3d21f91bdeba
Oct 30 21:24:05.885900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.885871    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:05.885999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.885943    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:05.885999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:05.885988    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:06.134851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:06.134766    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:06.140676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:06.140647    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:06.552655 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:06.552615    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" event={"ID":"3760152e-c736-402c-89b3-b862b8fd1193","Type":"ContainerStarted","Data":"eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa"}
Oct 30 21:24:06.552811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:06.552681    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" event={"ID":"3760152e-c736-402c-89b3-b862b8fd1193","Type":"ContainerStarted","Data":"064a94e6973e7a68c99f19f5e4fe3196145baca559935838e86a3d21f91bdeba"}
Oct 30 21:24:06.553016 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:06.552998    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:06.553964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:06.553943    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:24:06.671405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:06.671355    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" podStartSLOduration=1.671339353 podStartE2EDuration="1.671339353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:06.670329543 +0000 UTC m=+702.541448663" watchObservedRunningTime="2026-10-30 21:24:06.671339353 +0000 UTC m=+702.542458551"
Oct 30 21:24:06.748929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:06.748903    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:07.417265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:07.417238    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:07.421952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:07.421930    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:07.564678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:07.564640    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:07.564858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:07.564732    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:07.565549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:07.565525    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:07.565738 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:07.565663    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:13.777663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.777627    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"]
Oct 30 21:24:13.782392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.782372    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.831573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.831528    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"]
Oct 30 21:24:13.835160 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.835140    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:13.856413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.856386    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-9ac5c5-openshift-default-dockercfg-bnx6p\""
Oct 30 21:24:13.874246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874197    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.874345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874304    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.874345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874330    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.874417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874348    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27bmt\" (UniqueName: \"kubernetes.io/projected/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-kube-api-access-27bmt\") pod \"payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt\" (UID: \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:13.874417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874374    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.874519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874502    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.874553 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874529    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf59d\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-kube-api-access-rf59d\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.874586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874559    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.874663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874649    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt\" (UID: \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:13.874698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874677    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.874729 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.874709    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.883368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.883344    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"]
Oct 30 21:24:13.886585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.886563    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-847c62\""
Oct 30 21:24:13.891940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.891918    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"]
Oct 30 21:24:13.936525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.934244    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-847c62-dockercfg-bkcln\""
Oct 30 21:24:13.982182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982125    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.982182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982181    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982231    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27bmt\" (UniqueName: \"kubernetes.io/projected/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-kube-api-access-27bmt\") pod \"payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt\" (UID: \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:13.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982272    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982336    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982363    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf59d\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-kube-api-access-rf59d\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982397    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982463    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt\" (UID: \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:13.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982496    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.982866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982546    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.982866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.982602    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.983044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.983019    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.984360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.984081    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.984771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.984748    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.985386 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.985363    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt\" (UID: \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:13.986008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.985977    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.986101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.986087    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.986474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.986456    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"]
Oct 30 21:24:13.992562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.992527    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:13.992934 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.992917    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:13.993261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:13.993240    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:14.052140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.052058    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"]
Oct 30 21:24:14.053627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.053601    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf59d\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-kube-api-access-rf59d\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:14.073733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.073706    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27bmt\" (UniqueName: \"kubernetes.io/projected/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-kube-api-access-27bmt\") pod \"payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt\" (UID: \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:14.073903 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.073731    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:14.100496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.086853    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g\" (UID: \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:14.100496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.087042    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hb2g\" (UniqueName: \"kubernetes.io/projected/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-kube-api-access-5hb2g\") pod \"payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g\" (UID: \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:14.100822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.100801    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:14.153231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.148438    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:14.193766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.193736    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hb2g\" (UniqueName: \"kubernetes.io/projected/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-kube-api-access-5hb2g\") pod \"payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g\" (UID: \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:14.193905 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.193805    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g\" (UID: \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:14.194569 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.194547    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g\" (UID: \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:14.252083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.251541    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hb2g\" (UniqueName: \"kubernetes.io/projected/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-kube-api-access-5hb2g\") pod \"payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g\" (UID: \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:14.322674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.322596    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:14.542871 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.542835    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"]
Oct 30 21:24:14.582919 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:14.582882    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f382dea_dd70_4ebc_accb_a9bab6fc577f.slice/crio-08549e1a943aba771f18544784e93e2024364e85db5c700b4bc3d7303758931f WatchSource:0}: Error finding container 08549e1a943aba771f18544784e93e2024364e85db5c700b4bc3d7303758931f: Status 404 returned error can't find the container with id 08549e1a943aba771f18544784e93e2024364e85db5c700b4bc3d7303758931f
Oct 30 21:24:14.585034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.585004    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:14.585087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.585069    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:14.585120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.585094    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:14.593464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.593445    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"]
Oct 30 21:24:14.596188 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:14.596167    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7142d6be_6d9e_4802_8ce4_e0a2a29b187b.slice/crio-e8867f8e27f1a0d83d04c2b973914acfe2c21f7915fc6d6994a7de36cd1286ad WatchSource:0}: Error finding container e8867f8e27f1a0d83d04c2b973914acfe2c21f7915fc6d6994a7de36cd1286ad: Status 404 returned error can't find the container with id e8867f8e27f1a0d83d04c2b973914acfe2c21f7915fc6d6994a7de36cd1286ad
Oct 30 21:24:14.599279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.599251    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" event={"ID":"1f382dea-dd70-4ebc-accb-a9bab6fc577f","Type":"ContainerStarted","Data":"08549e1a943aba771f18544784e93e2024364e85db5c700b4bc3d7303758931f"}
Oct 30 21:24:14.695140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:14.694973    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"]
Oct 30 21:24:14.698636 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:14.698602    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94afe0a5_f17a_4c0d_9cae_ee100ce85d11.slice/crio-13e50d59ffccf842025f3dcbfc7f86fcc8922ecb15616f7b94819562cfbdae2c WatchSource:0}: Error finding container 13e50d59ffccf842025f3dcbfc7f86fcc8922ecb15616f7b94819562cfbdae2c: Status 404 returned error can't find the container with id 13e50d59ffccf842025f3dcbfc7f86fcc8922ecb15616f7b94819562cfbdae2c
Oct 30 21:24:15.605350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.605317    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" event={"ID":"1f382dea-dd70-4ebc-accb-a9bab6fc577f","Type":"ContainerStarted","Data":"e105ef17d6ae46cb10254c555c781b7a4472cd0809c1d506dc637237b0d507f8"}
Oct 30 21:24:15.607003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.606978    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g" event={"ID":"94afe0a5-f17a-4c0d-9cae-ee100ce85d11","Type":"ContainerStarted","Data":"618e018327980aa9fbdb45fad6f24120dbc03f1feaa5574d142ecc66cec957a4"}
Oct 30 21:24:15.607148 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.607019    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g" event={"ID":"94afe0a5-f17a-4c0d-9cae-ee100ce85d11","Type":"ContainerStarted","Data":"13e50d59ffccf842025f3dcbfc7f86fcc8922ecb15616f7b94819562cfbdae2c"}
Oct 30 21:24:15.607148 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.607046    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:15.608362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.608341    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt" event={"ID":"7142d6be-6d9e-4802-8ce4-e0a2a29b187b","Type":"ContainerStarted","Data":"d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b"}
Oct 30 21:24:15.608455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.608377    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt" event={"ID":"7142d6be-6d9e-4802-8ce4-e0a2a29b187b","Type":"ContainerStarted","Data":"e8867f8e27f1a0d83d04c2b973914acfe2c21f7915fc6d6994a7de36cd1286ad"}
Oct 30 21:24:15.608455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.608410    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:15.725344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.725295    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" podStartSLOduration=2.725277629 podStartE2EDuration="2.725277629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:15.724264097 +0000 UTC m=+711.595383211" watchObservedRunningTime="2026-10-30 21:24:15.725277629 +0000 UTC m=+711.596396749"
Oct 30 21:24:15.859299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.859004    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt" podStartSLOduration=2.858983804 podStartE2EDuration="2.858983804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:15.849574472 +0000 UTC m=+711.720693576" watchObservedRunningTime="2026-10-30 21:24:15.858983804 +0000 UTC m=+711.730102924"
Oct 30 21:24:15.975918 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:15.975859    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g" podStartSLOduration=2.975838305 podStartE2EDuration="2.975838305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:15.974870815 +0000 UTC m=+711.845989935" watchObservedRunningTime="2026-10-30 21:24:15.975838305 +0000 UTC m=+711.846957424"
Oct 30 21:24:16.101727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:16.101694    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:16.106452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:16.106428    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:16.613618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:16.613577    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:16.614304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:16.614287    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:24:21.119360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.119324    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"]
Oct 30 21:24:21.123397 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.123378    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:21.179913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.179885    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"]
Oct 30 21:24:21.181327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.181304    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16e6d78c-7d7b-4c62-be32-5808590e1345-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5\" (UID: \"16e6d78c-7d7b-4c62-be32-5808590e1345\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:21.181481 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.181461    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq4vp\" (UniqueName: \"kubernetes.io/projected/16e6d78c-7d7b-4c62-be32-5808590e1345-kube-api-access-pq4vp\") pod \"payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5\" (UID: \"16e6d78c-7d7b-4c62-be32-5808590e1345\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:21.228227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.226840    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-be9451\""
Oct 30 21:24:21.228227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.227016    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-be9451-dockercfg-bk68t\""
Oct 30 21:24:21.278082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.278046    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"]
Oct 30 21:24:21.281720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.281699    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:21.282190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.282170    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16e6d78c-7d7b-4c62-be32-5808590e1345-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5\" (UID: \"16e6d78c-7d7b-4c62-be32-5808590e1345\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:21.282289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.282276    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4vp\" (UniqueName: \"kubernetes.io/projected/16e6d78c-7d7b-4c62-be32-5808590e1345-kube-api-access-pq4vp\") pod \"payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5\" (UID: \"16e6d78c-7d7b-4c62-be32-5808590e1345\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:21.282838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.282820    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16e6d78c-7d7b-4c62-be32-5808590e1345-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5\" (UID: \"16e6d78c-7d7b-4c62-be32-5808590e1345\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:21.323003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.322970    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"]
Oct 30 21:24:21.338861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.338839    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq4vp\" (UniqueName: \"kubernetes.io/projected/16e6d78c-7d7b-4c62-be32-5808590e1345-kube-api-access-pq4vp\") pod \"payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5\" (UID: \"16e6d78c-7d7b-4c62-be32-5808590e1345\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:21.383497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.383392    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9679d46-0e10-4523-af31-7362a4871af7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4\" (UID: \"b9679d46-0e10-4523-af31-7362a4871af7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:21.383497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.383450    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7bzl\" (UniqueName: \"kubernetes.io/projected/b9679d46-0e10-4523-af31-7362a4871af7-kube-api-access-x7bzl\") pod \"payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4\" (UID: \"b9679d46-0e10-4523-af31-7362a4871af7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:21.436167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.436140    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:21.486021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.485991    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9679d46-0e10-4523-af31-7362a4871af7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4\" (UID: \"b9679d46-0e10-4523-af31-7362a4871af7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:21.486191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.486060    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7bzl\" (UniqueName: \"kubernetes.io/projected/b9679d46-0e10-4523-af31-7362a4871af7-kube-api-access-x7bzl\") pod \"payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4\" (UID: \"b9679d46-0e10-4523-af31-7362a4871af7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:21.486778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.486758    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9679d46-0e10-4523-af31-7362a4871af7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4\" (UID: \"b9679d46-0e10-4523-af31-7362a4871af7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:21.542592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.542566    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7bzl\" (UniqueName: \"kubernetes.io/projected/b9679d46-0e10-4523-af31-7362a4871af7-kube-api-access-x7bzl\") pod \"payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4\" (UID: \"b9679d46-0e10-4523-af31-7362a4871af7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:21.595838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.595809    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:21.810097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.810073    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"]
Oct 30 21:24:21.812564 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:21.812538    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e6d78c_7d7b_4c62_be32_5808590e1345.slice/crio-0b4248b77ed33550b21477d0e458626a729f7701b5b2f57a7772c5557a686f75 WatchSource:0}: Error finding container 0b4248b77ed33550b21477d0e458626a729f7701b5b2f57a7772c5557a686f75: Status 404 returned error can't find the container with id 0b4248b77ed33550b21477d0e458626a729f7701b5b2f57a7772c5557a686f75
Oct 30 21:24:21.972122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:21.972091    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"]
Oct 30 21:24:21.975278 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:21.975249    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9679d46_0e10_4523_af31_7362a4871af7.slice/crio-e0d77e2789981ecd8b92d40760320618e67880db7093479cccfd5c45da2f2f29 WatchSource:0}: Error finding container e0d77e2789981ecd8b92d40760320618e67880db7093479cccfd5c45da2f2f29: Status 404 returned error can't find the container with id e0d77e2789981ecd8b92d40760320618e67880db7093479cccfd5c45da2f2f29
Oct 30 21:24:22.353824 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:24:22.353731    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:24:22.646833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:22.646742    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5" event={"ID":"16e6d78c-7d7b-4c62-be32-5808590e1345","Type":"ContainerStarted","Data":"1b0178b3915c4284ef636216e222be7830d4a9d16854bbbb632e508f7724e62d"}
Oct 30 21:24:22.646833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:22.646797    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5" event={"ID":"16e6d78c-7d7b-4c62-be32-5808590e1345","Type":"ContainerStarted","Data":"0b4248b77ed33550b21477d0e458626a729f7701b5b2f57a7772c5557a686f75"}
Oct 30 21:24:22.647057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:22.646847    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:22.648417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:22.648396    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4" event={"ID":"b9679d46-0e10-4523-af31-7362a4871af7","Type":"ContainerStarted","Data":"23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd"}
Oct 30 21:24:22.648522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:22.648430    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4" event={"ID":"b9679d46-0e10-4523-af31-7362a4871af7","Type":"ContainerStarted","Data":"e0d77e2789981ecd8b92d40760320618e67880db7093479cccfd5c45da2f2f29"}
Oct 30 21:24:22.648522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:22.648460    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:22.760790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:22.760734    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5" podStartSLOduration=1.7607202979999999 podStartE2EDuration="1.760720298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:22.759297002 +0000 UTC m=+718.630416121" watchObservedRunningTime="2026-10-30 21:24:22.760720298 +0000 UTC m=+718.631839416"
Oct 30 21:24:22.884228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:22.883064    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4" podStartSLOduration=1.8830447750000001 podStartE2EDuration="1.883044775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:22.882474493 +0000 UTC m=+718.753593613" watchObservedRunningTime="2026-10-30 21:24:22.883044775 +0000 UTC m=+718.754163889"
Oct 30 21:24:23.795829 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:23.795800    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"]
Oct 30 21:24:23.799674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:23.799653    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:23.855967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:23.855939    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"]
Oct 30 21:24:23.903361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:23.903329    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-38b832\""
Oct 30 21:24:23.903361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:23.903348    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-38b832-dockercfg-28dbm\""
Oct 30 21:24:23.917883 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:23.917853    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sth8c\" (UniqueName: \"kubernetes.io/projected/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-kube-api-access-sth8c\") pod \"payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24\" (UID: \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:23.918068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:23.918046    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24\" (UID: \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:24.018696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.018667    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24\" (UID: \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:24.018833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.018720    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sth8c\" (UniqueName: \"kubernetes.io/projected/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-kube-api-access-sth8c\") pod \"payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24\" (UID: \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:24.019356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.019338    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24\" (UID: \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:24.074991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.074922    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sth8c\" (UniqueName: \"kubernetes.io/projected/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-kube-api-access-sth8c\") pod \"payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24\" (UID: \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:24.117929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.117904    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:24.490415 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.490393    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"]
Oct 30 21:24:24.492367 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:24.492345    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5f2a0f_9f81_4f88_8e65_bb4499139f87.slice/crio-8c28da515932a9f288f3411f70df93e492b3646cd43837246a2baecd0b777e91 WatchSource:0}: Error finding container 8c28da515932a9f288f3411f70df93e492b3646cd43837246a2baecd0b777e91: Status 404 returned error can't find the container with id 8c28da515932a9f288f3411f70df93e492b3646cd43837246a2baecd0b777e91
Oct 30 21:24:24.660397 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.660363    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24" event={"ID":"bb5f2a0f-9f81-4f88-8e65-bb4499139f87","Type":"ContainerStarted","Data":"f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e"}
Oct 30 21:24:24.660558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.660414    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24" event={"ID":"bb5f2a0f-9f81-4f88-8e65-bb4499139f87","Type":"ContainerStarted","Data":"8c28da515932a9f288f3411f70df93e492b3646cd43837246a2baecd0b777e91"}
Oct 30 21:24:24.660558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.660465    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:24.776565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:24.776472    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24" podStartSLOduration=1.77645836 podStartE2EDuration="1.77645836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:24.775918641 +0000 UTC m=+720.647037761" watchObservedRunningTime="2026-10-30 21:24:24.77645836 +0000 UTC m=+720.647577478"
Oct 30 21:24:26.616079 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:26.616054    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:24:26.616482 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:26.616117    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:24:28.381509 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.381478    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"]
Oct 30 21:24:28.385731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.385709    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:28.436920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.436891    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-8ce31a\""
Oct 30 21:24:28.464338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.464302    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22504f51-bb1e-499a-b14a-cd422f024088-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j\" (UID: \"22504f51-bb1e-499a-b14a-cd422f024088\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:28.464517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.464351    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cts7m\" (UniqueName: \"kubernetes.io/projected/22504f51-bb1e-499a-b14a-cd422f024088-kube-api-access-cts7m\") pod \"payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j\" (UID: \"22504f51-bb1e-499a-b14a-cd422f024088\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:28.487310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.487194    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-8ce31a-dockercfg-m9s87\""
Oct 30 21:24:28.535605 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.535520    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"]
Oct 30 21:24:28.566133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.566099    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cts7m\" (UniqueName: \"kubernetes.io/projected/22504f51-bb1e-499a-b14a-cd422f024088-kube-api-access-cts7m\") pod \"payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j\" (UID: \"22504f51-bb1e-499a-b14a-cd422f024088\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:28.566347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.566326    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22504f51-bb1e-499a-b14a-cd422f024088-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j\" (UID: \"22504f51-bb1e-499a-b14a-cd422f024088\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:28.567517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.567491    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22504f51-bb1e-499a-b14a-cd422f024088-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j\" (UID: \"22504f51-bb1e-499a-b14a-cd422f024088\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:28.622813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.622780    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cts7m\" (UniqueName: \"kubernetes.io/projected/22504f51-bb1e-499a-b14a-cd422f024088-kube-api-access-cts7m\") pod \"payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j\" (UID: \"22504f51-bb1e-499a-b14a-cd422f024088\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:28.699007 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:28.698982    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:29.086534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:29.086352    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"]
Oct 30 21:24:29.089533 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:29.089505    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22504f51_bb1e_499a_b14a_cd422f024088.slice/crio-92f721b42fad273167c792f8f43dc1f54a0168db6118630fa295674c5f210136 WatchSource:0}: Error finding container 92f721b42fad273167c792f8f43dc1f54a0168db6118630fa295674c5f210136: Status 404 returned error can't find the container with id 92f721b42fad273167c792f8f43dc1f54a0168db6118630fa295674c5f210136
Oct 30 21:24:29.687872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:29.687838    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j" event={"ID":"22504f51-bb1e-499a-b14a-cd422f024088","Type":"ContainerStarted","Data":"6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88"}
Oct 30 21:24:29.688323 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:29.687887    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j" event={"ID":"22504f51-bb1e-499a-b14a-cd422f024088","Type":"ContainerStarted","Data":"92f721b42fad273167c792f8f43dc1f54a0168db6118630fa295674c5f210136"}
Oct 30 21:24:29.688323 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:29.687937    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:29.804099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:29.804049    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j" podStartSLOduration=1.804033368 podStartE2EDuration="1.804033368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:29.802850103 +0000 UTC m=+725.673969227" watchObservedRunningTime="2026-10-30 21:24:29.804033368 +0000 UTC m=+725.675152486"
Oct 30 21:24:33.221900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.221867    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"]
Oct 30 21:24:33.225590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.225566    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:33.280180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.280102    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-9ac5c5\""
Oct 30 21:24:33.280913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.280887    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-9ac5c5-dockercfg-8sq6v\""
Oct 30 21:24:33.283145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.283121    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"]
Oct 30 21:24:33.328470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.328444    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfcc7\" (UniqueName: \"kubernetes.io/projected/e244748d-aecf-4ab3-973f-d0eae1307c1c-kube-api-access-hfcc7\") pod \"payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c\" (UID: \"e244748d-aecf-4ab3-973f-d0eae1307c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:33.328614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.328568    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e244748d-aecf-4ab3-973f-d0eae1307c1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c\" (UID: \"e244748d-aecf-4ab3-973f-d0eae1307c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:33.368738 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.368532    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"]
Oct 30 21:24:33.431310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.431274    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfcc7\" (UniqueName: \"kubernetes.io/projected/e244748d-aecf-4ab3-973f-d0eae1307c1c-kube-api-access-hfcc7\") pod \"payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c\" (UID: \"e244748d-aecf-4ab3-973f-d0eae1307c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:33.431507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.431382    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e244748d-aecf-4ab3-973f-d0eae1307c1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c\" (UID: \"e244748d-aecf-4ab3-973f-d0eae1307c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:33.432132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.432108    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"]
Oct 30 21:24:33.432259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.432223    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e244748d-aecf-4ab3-973f-d0eae1307c1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c\" (UID: \"e244748d-aecf-4ab3-973f-d0eae1307c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:33.432328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.432272    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:33.488462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.488397    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfcc7\" (UniqueName: \"kubernetes.io/projected/e244748d-aecf-4ab3-973f-d0eae1307c1c-kube-api-access-hfcc7\") pod \"payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c\" (UID: \"e244748d-aecf-4ab3-973f-d0eae1307c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:33.542421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.537448    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:33.542421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.538740    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aead0097-88ae-4970-81c2-f856c8000169-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk\" (UID: \"aead0097-88ae-4970-81c2-f856c8000169\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:33.542421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.538809    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx69q\" (UniqueName: \"kubernetes.io/projected/aead0097-88ae-4970-81c2-f856c8000169-kube-api-access-rx69q\") pod \"payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk\" (UID: \"aead0097-88ae-4970-81c2-f856c8000169\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:33.639679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.639645    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx69q\" (UniqueName: \"kubernetes.io/projected/aead0097-88ae-4970-81c2-f856c8000169-kube-api-access-rx69q\") pod \"payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk\" (UID: \"aead0097-88ae-4970-81c2-f856c8000169\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:33.639801 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.639788    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aead0097-88ae-4970-81c2-f856c8000169-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk\" (UID: \"aead0097-88ae-4970-81c2-f856c8000169\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:33.640678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.640650    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aead0097-88ae-4970-81c2-f856c8000169-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk\" (UID: \"aead0097-88ae-4970-81c2-f856c8000169\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:33.679042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.678906    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:24:33.679271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.679052    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:24:33.698162 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.698130    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx69q\" (UniqueName: \"kubernetes.io/projected/aead0097-88ae-4970-81c2-f856c8000169-kube-api-access-rx69q\") pod \"payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk\" (UID: \"aead0097-88ae-4970-81c2-f856c8000169\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:33.748579 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.748514    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:33.945183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:33.945154    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"]
Oct 30 21:24:33.945725 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:33.945704    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode244748d_aecf_4ab3_973f_d0eae1307c1c.slice/crio-e394752cec240f812ee94654c214c2dca0b50a921546ce6aaa206f157245d4a0 WatchSource:0}: Error finding container e394752cec240f812ee94654c214c2dca0b50a921546ce6aaa206f157245d4a0: Status 404 returned error can't find the container with id e394752cec240f812ee94654c214c2dca0b50a921546ce6aaa206f157245d4a0
Oct 30 21:24:34.166346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.163539    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"]
Oct 30 21:24:34.171867 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:34.171836    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaead0097_88ae_4970_81c2_f856c8000169.slice/crio-dd8df5f11d05a4f426b517a2f4929bb60be8d65af0282703f8a8445fc998b914 WatchSource:0}: Error finding container dd8df5f11d05a4f426b517a2f4929bb60be8d65af0282703f8a8445fc998b914: Status 404 returned error can't find the container with id dd8df5f11d05a4f426b517a2f4929bb60be8d65af0282703f8a8445fc998b914
Oct 30 21:24:34.722525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.722487    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk" event={"ID":"aead0097-88ae-4970-81c2-f856c8000169","Type":"ContainerStarted","Data":"4882cb1318d13148f4564a602ad4cee02021337a646f2b5d3050a075a1c58620"}
Oct 30 21:24:34.722968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.722551    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk" event={"ID":"aead0097-88ae-4970-81c2-f856c8000169","Type":"ContainerStarted","Data":"dd8df5f11d05a4f426b517a2f4929bb60be8d65af0282703f8a8445fc998b914"}
Oct 30 21:24:34.722968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.722712    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:34.724062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.724036    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c" event={"ID":"e244748d-aecf-4ab3-973f-d0eae1307c1c","Type":"ContainerStarted","Data":"8b2d7c23c4d6636069373318984d16728594fe35bb79b033ceea3174a590ca6d"}
Oct 30 21:24:34.724182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.724078    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c" event={"ID":"e244748d-aecf-4ab3-973f-d0eae1307c1c","Type":"ContainerStarted","Data":"e394752cec240f812ee94654c214c2dca0b50a921546ce6aaa206f157245d4a0"}
Oct 30 21:24:34.724182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.724105    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:34.838052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.838003    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk" podStartSLOduration=1.837989125 podStartE2EDuration="1.837989125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:34.836621089 +0000 UTC m=+730.707740208" watchObservedRunningTime="2026-10-30 21:24:34.837989125 +0000 UTC m=+730.709108243"
Oct 30 21:24:34.976292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:34.971857    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c" podStartSLOduration=1.971838685 podStartE2EDuration="1.971838685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:34.970278788 +0000 UTC m=+730.841397912" watchObservedRunningTime="2026-10-30 21:24:34.971838685 +0000 UTC m=+730.842957805"
Oct 30 21:24:35.668409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:35.668380    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:24:37.882665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:37.882620    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": read tcp 10.134.0.2:57544->10.134.0.42:15021: read: connection reset by peer" start-of-body=
Oct 30 21:24:37.885107 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:37.882708    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": read tcp 10.134.0.2:57544->10.134.0.42:15021: read: connection reset by peer"
Oct 30 21:24:38.751063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:38.751030    5786 generic.go:356] "Generic (PLEG): container finished" podID="ca508309-2ab8-420f-a9d1-f26963632e25" containerID="3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3" exitCode=0
Oct 30 21:24:38.751295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:38.751101    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" event={"ID":"ca508309-2ab8-420f-a9d1-f26963632e25","Type":"ContainerDied","Data":"3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3"}
Oct 30 21:24:38.774229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:38.751595    5786 scope.go:122] "RemoveContainer" containerID="3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3"
Oct 30 21:24:38.777019 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:38.776977    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:38.777111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:38.777067    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:38.777111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:38.777101    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:39.804099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:39.804063    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" event={"ID":"ca508309-2ab8-420f-a9d1-f26963632e25","Type":"ContainerStarted","Data":"ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f"}
Oct 30 21:24:40.386303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.386073    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"]
Oct 30 21:24:40.406653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.406618    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:40.442465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.442436    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"]
Oct 30 21:24:40.459528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.459498    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-9f52c3\""
Oct 30 21:24:40.479302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.459860    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-9f52c3-dockercfg-smfz4\""
Oct 30 21:24:40.514401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.505723    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s\" (UID: \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:40.529345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.529305    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9gcv\" (UniqueName: \"kubernetes.io/projected/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-kube-api-access-k9gcv\") pod \"payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s\" (UID: \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:40.631371 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.631336    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s\" (UID: \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:40.631538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.631382    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9gcv\" (UniqueName: \"kubernetes.io/projected/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-kube-api-access-k9gcv\") pod \"payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s\" (UID: \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:40.632616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.632593    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s\" (UID: \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:40.704687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.704552    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9gcv\" (UniqueName: \"kubernetes.io/projected/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-kube-api-access-k9gcv\") pod \"payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s\" (UID: \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:40.706583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.706561    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:24:40.721540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.721520    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:40.744822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.744760    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:40.744991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.744973    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:24:40.745071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:40.745020    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:24:41.160287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:41.157736    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"]
Oct 30 21:24:41.744425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:41.744339    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:24:41.744425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:41.744392    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:24:41.820678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:41.820643    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s" event={"ID":"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe","Type":"ContainerStarted","Data":"d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9"}
Oct 30 21:24:41.820678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:41.820694    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s" event={"ID":"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe","Type":"ContainerStarted","Data":"05d3c85b711f161a0deff724a2757cc0dfeaedb199dbd7b798e71e1ce50e855c"}
Oct 30 21:24:41.820933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:41.820755    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:41.938061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:41.937997    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s" podStartSLOduration=1.9379783910000001 podStartE2EDuration="1.937978391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:41.937675085 +0000 UTC m=+737.808794202" watchObservedRunningTime="2026-10-30 21:24:41.937978391 +0000 UTC m=+737.809097511"
Oct 30 21:24:42.744040 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:42.744005    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:24:42.744524 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:42.744061    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:24:43.743993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:43.743953    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:24:43.744194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:43.744012    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:24:44.744086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:44.744048    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:24:44.744449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:44.744102    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:24:45.733195 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.733168    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:24:45.733419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.733254    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:24:45.747391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.747370    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:45.747680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.747626    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:45.748170 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.748155    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:24:45.845713 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.845683    5786 generic.go:356] "Generic (PLEG): container finished" podID="3760152e-c736-402c-89b3-b862b8fd1193" containerID="eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa" exitCode=0
Oct 30 21:24:45.845871 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.845757    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" event={"ID":"3760152e-c736-402c-89b3-b862b8fd1193","Type":"ContainerDied","Data":"eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa"}
Oct 30 21:24:45.846632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.846553    5786 scope.go:122] "RemoveContainer" containerID="eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa"
Oct 30 21:24:45.848466 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.848443    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:45.848529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.848508    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:45.848563 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:45.848551    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:46.871531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:46.871490    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" event={"ID":"3760152e-c736-402c-89b3-b862b8fd1193","Type":"ContainerStarted","Data":"c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88"}
Oct 30 21:24:47.417361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:47.417334    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:47.417550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:47.417486    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:24:47.417550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:47.417528    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 21:24:48.417311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:48.417277    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:24:48.417715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:48.417325    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 21:24:49.417811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:49.417771    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:24:49.418266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:49.417827    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 21:24:50.417465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:50.417427    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:24:50.417659 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:50.417474    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 21:24:51.041300 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.041261    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"]
Oct 30 21:24:51.046657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.046636    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.118698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.118665    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r2xv\" (UniqueName: \"kubernetes.io/projected/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-kube-api-access-6r2xv\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.118884 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.118718    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.118884 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.118829    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-metrics-certs\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.118884 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.118868    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.165233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.151318    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:24:51.165233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.158780    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"]
Oct 30 21:24:51.165233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.159002    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:24:51.165233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.159037    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-9ac5c5-serving-cert\""
Oct 30 21:24:51.165233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.159111    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:24:51.200540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.200491    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wqlw4\""
Oct 30 21:24:51.219518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.219485    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r2xv\" (UniqueName: \"kubernetes.io/projected/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-kube-api-access-6r2xv\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.219699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.219537    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.219699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.219562    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-metrics-certs\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.219699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.219582    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.219699 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:24:51.219699    5786 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-serving-cert: secret "maas-api-e2e-worker-w6-9ac5c5-serving-cert" not found
Oct 30 21:24:51.219919 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:24:51.219778    5786 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls podName:c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b nodeName:}" failed. No retries permitted until 2026-10-30 21:24:51.719762829 +0000 UTC m=+747.590881939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls") pod "maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc" (UID: "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b") : secret "maas-api-e2e-worker-w6-9ac5c5-serving-cert" not found
Oct 30 21:24:51.220577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.220552    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.222912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.222886    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-metrics-certs\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.302435 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.298580    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r2xv\" (UniqueName: \"kubernetes.io/projected/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-kube-api-access-6r2xv\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.418646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.418599    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:24:51.418847 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.418672    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 21:24:51.725546 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.725506    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.728047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.728021    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:51.973462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:51.973427    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:52.374902 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:24:52.374818    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:24:52.382465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:52.382435    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"]
Oct 30 21:24:52.382921 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:24:52.382900    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0720946_3b9c_4a5e_b3c8_d3e99d5fa11b.slice/crio-73fea11041d0dcc491cafc14cc8487f478f041fb18e7d5db8fd2dbd8100dee00 WatchSource:0}: Error finding container 73fea11041d0dcc491cafc14cc8487f478f041fb18e7d5db8fd2dbd8100dee00: Status 404 returned error can't find the container with id 73fea11041d0dcc491cafc14cc8487f478f041fb18e7d5db8fd2dbd8100dee00
Oct 30 21:24:52.420719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:52.420694    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:52.421089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:52.421074    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:52.421811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:52.421794    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:24:52.829387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:52.829360    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:24:52.921045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:52.921005    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc" event={"ID":"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b","Type":"ContainerStarted","Data":"73fea11041d0dcc491cafc14cc8487f478f041fb18e7d5db8fd2dbd8100dee00"}
Oct 30 21:24:54.933266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:54.933226    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc" event={"ID":"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b","Type":"ContainerStarted","Data":"b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b"}
Oct 30 21:24:54.933698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:54.933404    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:24:55.062023 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:55.061973    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc" podStartSLOduration=1.8116642 podStartE2EDuration="4.061956512s" totalImagesPullingTime="2.250292312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:55.061319208 +0000 UTC m=+750.932438327" watchObservedRunningTime="2026-10-30 21:24:55.061956512 +0000 UTC m=+750.933075630"
Oct 30 21:24:58.956092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:58.956058    5786 generic.go:356] "Generic (PLEG): container finished" podID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerID="fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e" exitCode=0
Oct 30 21:24:58.956664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:58.956135    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" event={"ID":"2a36ca42-23dc-46ef-a105-cf2d9ee227f0","Type":"ContainerDied","Data":"fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e"}
Oct 30 21:24:58.956664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:58.956628    5786 scope.go:122] "RemoveContainer" containerID="fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e"
Oct 30 21:24:58.957923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:58.957898    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:58.957985 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:58.957962    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:58.958018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:58.957990    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:24:59.981824 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:24:59.981791    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" event={"ID":"2a36ca42-23dc-46ef-a105-cf2d9ee227f0","Type":"ContainerStarted","Data":"ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f"}
Oct 30 21:25:00.135720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:00.135687    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:25:00.135902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:00.135855    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:25:00.135947 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:00.135899    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:25:00.956405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:00.956368    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:25:01.135314 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:01.135282    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:25:01.135684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:01.135330    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:25:02.135807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:02.135771    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:25:02.136218 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:02.135818    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:25:03.135051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:03.135003    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:25:03.135247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:03.135063    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:25:04.135525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:04.135489    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:25:04.136060 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:04.135551    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:25:05.239039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:05.239003    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:25:05.240123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:05.240082    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:25:05.240954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:05.240934    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:25:10.061879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:10.061845    5786 generic.go:356] "Generic (PLEG): container finished" podID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerID="e105ef17d6ae46cb10254c555c781b7a4472cd0809c1d506dc637237b0d507f8" exitCode=0
Oct 30 21:25:10.062331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:10.061923    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" event={"ID":"1f382dea-dd70-4ebc-accb-a9bab6fc577f","Type":"ContainerDied","Data":"e105ef17d6ae46cb10254c555c781b7a4472cd0809c1d506dc637237b0d507f8"}
Oct 30 21:25:10.062331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:10.062328    5786 scope.go:122] "RemoveContainer" containerID="e105ef17d6ae46cb10254c555c781b7a4472cd0809c1d506dc637237b0d507f8"
Oct 30 21:25:10.063629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:10.063599    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:25:10.063739 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:10.063679    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:25:10.063739 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:10.063727    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:25:11.082625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:11.082583    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" event={"ID":"1f382dea-dd70-4ebc-accb-a9bab6fc577f","Type":"ContainerStarted","Data":"c5caabc703a8a4260582d14f092027e58e388fcb8931f9c4d81c229799aebcec"}
Oct 30 21:25:11.101656 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:11.101633    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:25:11.101804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:11.101776    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 21:25:11.101861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:11.101821    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 21:25:12.102650 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:12.102614    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 21:25:12.103304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:12.103269    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 21:25:13.101861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:13.101826    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 21:25:13.102039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:13.101887    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 21:25:14.101704 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:14.101666    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 21:25:14.102098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:14.101718    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 21:25:15.101512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:15.101483    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 21:25:15.101704 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:15.101538    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 21:25:16.105304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:16.105276    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:25:16.105910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:16.105637    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:25:16.106368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:16.106350    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:25:22.355508 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:25:22.355472    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:25:32.096702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.096669    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"]
Oct 30 21:25:32.101833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.101812    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.132147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.132119    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbhfr\" (UniqueName: \"kubernetes.io/projected/ed6323c9-8489-4ad1-86ed-0c588083ba26-kube-api-access-gbhfr\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.132325 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.132154    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed6323c9-8489-4ad1-86ed-0c588083ba26-tls-certs\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.132373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.132359    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-tmp-dir\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.132424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.132406    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-home\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.132477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.132469    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-dshm\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.132531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.132519    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-model-cache\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.205322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.205293    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-38b832\"/\"default-dockercfg-289hb\""
Oct 30 21:25:32.205488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.205331    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-38b832\"/\"simulator-w2-38b832-kserve-self-signed-certs\""
Oct 30 21:25:32.207429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.207402    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"]
Oct 30 21:25:32.233360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233340    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbhfr\" (UniqueName: \"kubernetes.io/projected/ed6323c9-8489-4ad1-86ed-0c588083ba26-kube-api-access-gbhfr\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.233477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233370    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed6323c9-8489-4ad1-86ed-0c588083ba26-tls-certs\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.233477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233445    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-tmp-dir\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.233594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233482    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-home\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.233594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233507    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-dshm\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.233594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233579    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-model-cache\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.233896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233876    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-tmp-dir\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.233896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233883    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-home\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.233993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.233984    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-model-cache\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.235995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.235975    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-dshm\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.236339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.236322    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed6323c9-8489-4ad1-86ed-0c588083ba26-tls-certs\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.258652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.258496    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-38b832\"/\"kube-root-ca.crt\""
Oct 30 21:25:32.258652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.258619    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-38b832\"/\"openshift-service-ca.crt\""
Oct 30 21:25:32.290158 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.290124    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbhfr\" (UniqueName: \"kubernetes.io/projected/ed6323c9-8489-4ad1-86ed-0c588083ba26-kube-api-access-gbhfr\") pod \"simulator-w2-38b832-kserve-7766c47d5c-whp4r\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") " pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.413716 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.413696    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:32.791711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:32.791593    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"]
Oct 30 21:25:32.796263 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:25:32.796234    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded6323c9_8489_4ad1_86ed_0c588083ba26.slice/crio-2b51a92cd1a00debed34877cfb4c4559dd48d67444a7055d6cdfe315616b6a41 WatchSource:0}: Error finding container 2b51a92cd1a00debed34877cfb4c4559dd48d67444a7055d6cdfe315616b6a41: Status 404 returned error can't find the container with id 2b51a92cd1a00debed34877cfb4c4559dd48d67444a7055d6cdfe315616b6a41
Oct 30 21:25:33.284665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:33.284630    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" event={"ID":"ed6323c9-8489-4ad1-86ed-0c588083ba26","Type":"ContainerStarted","Data":"a1761962f7a8110950049d3b00744c699ad3cf1a022f1a92a8eda52f49634d46"}
Oct 30 21:25:33.284665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:33.284668    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" event={"ID":"ed6323c9-8489-4ad1-86ed-0c588083ba26","Type":"ContainerStarted","Data":"2b51a92cd1a00debed34877cfb4c4559dd48d67444a7055d6cdfe315616b6a41"}
Oct 30 21:25:33.401192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:33.401134    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podStartSLOduration=1.401119003 podStartE2EDuration="1.401119003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:33.399264636 +0000 UTC m=+789.270383753" watchObservedRunningTime="2026-10-30 21:25:33.401119003 +0000 UTC m=+789.272238121"
Oct 30 21:25:33.414531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:33.414504    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:42.414002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:42.413972    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:42.425747 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:42.425723    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:43.349150 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:43.349123    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:25:47.477231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.477169    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"]
Oct 30 21:25:47.481157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.481140    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.554697 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.554669    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-8ce31a\"/\"kube-root-ca.crt\""
Oct 30 21:25:47.583620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.583590    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-8ce31a\"/\"default-dockercfg-zprht\""
Oct 30 21:25:47.583620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.583595    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-8ce31a\"/\"openshift-service-ca.crt\""
Oct 30 21:25:47.583620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.583594    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-8ce31a\"/\"simulator-w1-8ce31a-kserve-self-signed-certs\""
Oct 30 21:25:47.585741 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.585723    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"]
Oct 30 21:25:47.609126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.609097    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bae5037c-823b-4c3d-9ec1-76ce3d142741-tls-certs\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.609312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.609130    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-dshm\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.609312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.609288    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-home\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.609421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.609333    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-model-cache\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.609473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.609455    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-tmp-dir\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.609512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.609488    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjmwt\" (UniqueName: \"kubernetes.io/projected/bae5037c-823b-4c3d-9ec1-76ce3d142741-kube-api-access-tjmwt\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.710809 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.710781    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-tmp-dir\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.710809 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.710816    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjmwt\" (UniqueName: \"kubernetes.io/projected/bae5037c-823b-4c3d-9ec1-76ce3d142741-kube-api-access-tjmwt\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.711080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.710869    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bae5037c-823b-4c3d-9ec1-76ce3d142741-tls-certs\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.711080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.710884    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-dshm\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.711080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.711031    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-home\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.711282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.711084    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-model-cache\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.711282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.711243    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-tmp-dir\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.711433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.711410    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-home\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.711475 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.711451    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-model-cache\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.713290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.713271    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-dshm\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.713534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.713518    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bae5037c-823b-4c3d-9ec1-76ce3d142741-tls-certs\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.767334 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.767239    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjmwt\" (UniqueName: \"kubernetes.io/projected/bae5037c-823b-4c3d-9ec1-76ce3d142741-kube-api-access-tjmwt\") pod \"simulator-w1-8ce31a-kserve-84c4974b9-xbh6r\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") " pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:47.794191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:47.794157    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:48.166676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:48.166650    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"]
Oct 30 21:25:48.168987 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:25:48.168962    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbae5037c_823b_4c3d_9ec1_76ce3d142741.slice/crio-dceefa7a2a0dd0357e8f39b58bb383d449f70f0841babc2c9bd05288f37765cb WatchSource:0}: Error finding container dceefa7a2a0dd0357e8f39b58bb383d449f70f0841babc2c9bd05288f37765cb: Status 404 returned error can't find the container with id dceefa7a2a0dd0357e8f39b58bb383d449f70f0841babc2c9bd05288f37765cb
Oct 30 21:25:48.372355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:48.372314    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" event={"ID":"bae5037c-823b-4c3d-9ec1-76ce3d142741","Type":"ContainerStarted","Data":"1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c"}
Oct 30 21:25:48.372355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:48.372357    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" event={"ID":"bae5037c-823b-4c3d-9ec1-76ce3d142741","Type":"ContainerStarted","Data":"dceefa7a2a0dd0357e8f39b58bb383d449f70f0841babc2c9bd05288f37765cb"}
Oct 30 21:25:48.488401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:48.488296    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podStartSLOduration=1.488280861 podStartE2EDuration="1.488280861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:48.486794755 +0000 UTC m=+804.357913875" watchObservedRunningTime="2026-10-30 21:25:48.488280861 +0000 UTC m=+804.359399979"
Oct 30 21:25:48.795268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:48.795163    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:52.354441 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:25:52.354406    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:25:57.794603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:57.794567    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:57.806337 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:57.806311    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:25:58.435389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:25:58.435366    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:26:10.836071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.836040    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"]
Oct 30 21:26:10.839935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.839912    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.892608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.892579    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tmp-dir\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.892766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.892625    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-model-cache\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.892766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.892649    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqv6p\" (UniqueName: \"kubernetes.io/projected/e82f1a15-4e36-4179-8e3b-653dbe03b86b-kube-api-access-xqv6p\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.892766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.892762    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tls-certs\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.892934 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.892817    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-dshm\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.892934 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.892868    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-home\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.944044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.944019    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-9ac5c5\"/\"openshift-service-ca.crt\""
Oct 30 21:26:10.944044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.944036    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-9ac5c5\"/\"kube-root-ca.crt\""
Oct 30 21:26:10.944330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.944313    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9ac5c5\"/\"default-dockercfg-8tlqp\""
Oct 30 21:26:10.944431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.944415    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9ac5c5\"/\"simulator-w6-9ac5c5-kserve-self-signed-certs\""
Oct 30 21:26:10.946114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.946096    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"]
Oct 30 21:26:10.994042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994019    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-dshm\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.994143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994049    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-home\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.994143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994104    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tmp-dir\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.994143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994132    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-model-cache\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.994321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994148    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqv6p\" (UniqueName: \"kubernetes.io/projected/e82f1a15-4e36-4179-8e3b-653dbe03b86b-kube-api-access-xqv6p\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.994321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994245    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tls-certs\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.994597 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994564    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-home\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.994597 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994581    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-model-cache\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.994728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.994609    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tmp-dir\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.996423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.996406    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-dshm\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:10.996662 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:10.996647    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tls-certs\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:11.082127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:11.079183    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqv6p\" (UniqueName: \"kubernetes.io/projected/e82f1a15-4e36-4179-8e3b-653dbe03b86b-kube-api-access-xqv6p\") pod \"simulator-w6-9ac5c5-kserve-7866684f8c-dmtht\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") " pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:11.151506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:11.151414    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:11.528550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:11.528524    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"]
Oct 30 21:26:11.531314 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:26:11.531279    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode82f1a15_4e36_4179_8e3b_653dbe03b86b.slice/crio-4cf76b0d691cef6a5f7a42a8467bdb7b43e8a4aeb50a4fe5a8003c088e506143 WatchSource:0}: Error finding container 4cf76b0d691cef6a5f7a42a8467bdb7b43e8a4aeb50a4fe5a8003c088e506143: Status 404 returned error can't find the container with id 4cf76b0d691cef6a5f7a42a8467bdb7b43e8a4aeb50a4fe5a8003c088e506143
Oct 30 21:26:12.499654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:12.499624    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" event={"ID":"e82f1a15-4e36-4179-8e3b-653dbe03b86b","Type":"ContainerStarted","Data":"8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c"}
Oct 30 21:26:12.500069 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:12.499660    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" event={"ID":"e82f1a15-4e36-4179-8e3b-653dbe03b86b","Type":"ContainerStarted","Data":"4cf76b0d691cef6a5f7a42a8467bdb7b43e8a4aeb50a4fe5a8003c088e506143"}
Oct 30 21:26:12.613629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:12.613577    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podStartSLOduration=2.613564249 podStartE2EDuration="2.613564249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:12.612812999 +0000 UTC m=+828.483932118" watchObservedRunningTime="2026-10-30 21:26:12.613564249 +0000 UTC m=+828.484683368"
Oct 30 21:26:13.152312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:13.152283    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:21.151611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:21.151575    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:21.163523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:21.163502    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:21.560925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:21.560901    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:26:22.358143 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:26:22.358108    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:26:26.167128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.167097    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"]
Oct 30 21:26:26.170668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.170649    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.222331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.222310    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-be9451\"/\"openshift-service-ca.crt\""
Oct 30 21:26:26.222459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.222433    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-be9451\"/\"kube-root-ca.crt\""
Oct 30 21:26:26.222532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.222516    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-be9451\"/\"premium-w0-be9451-kserve-self-signed-certs\""
Oct 30 21:26:26.257199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.257175    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-home\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.257354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.257200    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-dshm\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.257354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.257261    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-model-cache\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.257443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.257396    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tls-certs\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.257494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.257455    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n6c4\" (UniqueName: \"kubernetes.io/projected/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-kube-api-access-8n6c4\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.257534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.257519    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tmp-dir\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.273819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.273797    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"]
Oct 30 21:26:26.323652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.323622    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-be9451\"/\"default-dockercfg-xpqsf\""
Oct 30 21:26:26.358733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.358706    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-home\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.358907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.358742    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-dshm\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.358907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.358765    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-model-cache\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.358907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.358803    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tls-certs\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.358907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.358884    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n6c4\" (UniqueName: \"kubernetes.io/projected/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-kube-api-access-8n6c4\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.359136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.358918    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tmp-dir\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.359196 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.359157    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-home\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.359305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.359280    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-model-cache\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.359365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.359331    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tmp-dir\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.361405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.361381    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-dshm\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.361649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.361631    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tls-certs\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.416740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.416712    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n6c4\" (UniqueName: \"kubernetes.io/projected/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-kube-api-access-8n6c4\") pod \"premium-w0-be9451-kserve-79645b5648-8hwxr\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") " pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.484752 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.484699    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:26.887250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:26.887197    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"]
Oct 30 21:26:26.889371 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:26:26.889346    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61f2966e_e0d9_44b0_8ac5_69d7a678a0d8.slice/crio-28da04c2497cb189d2e816d816b995285cbeefc4edf42a20a8033b972a3ed8c5 WatchSource:0}: Error finding container 28da04c2497cb189d2e816d816b995285cbeefc4edf42a20a8033b972a3ed8c5: Status 404 returned error can't find the container with id 28da04c2497cb189d2e816d816b995285cbeefc4edf42a20a8033b972a3ed8c5
Oct 30 21:26:27.589539 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:27.589503    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" event={"ID":"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8","Type":"ContainerStarted","Data":"5722cfc26c816eb40f4ce7a2296e16a99a599311315151ec3c466d40bac78d17"}
Oct 30 21:26:27.589938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:27.589549    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" event={"ID":"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8","Type":"ContainerStarted","Data":"28da04c2497cb189d2e816d816b995285cbeefc4edf42a20a8033b972a3ed8c5"}
Oct 30 21:26:27.705484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:27.705438    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podStartSLOduration=1.7054247340000002 podStartE2EDuration="1.705424734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:27.704412453 +0000 UTC m=+843.575531574" watchObservedRunningTime="2026-10-30 21:26:27.705424734 +0000 UTC m=+843.576543852"
Oct 30 21:26:28.485444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:28.485405    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:36.485077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:36.485045    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:36.497482 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:36.497460    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:36.650245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:36.650192    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:26:48.085575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.085543    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"]
Oct 30 21:26:48.090063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.090039    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.197033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.197001    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f52c3\"/\"default-dockercfg-fmsr2\""
Oct 30 21:26:48.197264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.197143    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f52c3\"/\"simulator-w3-9f52c3-kserve-self-signed-certs\""
Oct 30 21:26:48.197264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.197242    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9f52c3\"/\"kube-root-ca.crt\""
Oct 30 21:26:48.197662 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.197314    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9f52c3\"/\"openshift-service-ca.crt\""
Oct 30 21:26:48.198602 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.198574    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-dshm\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.198798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.198612    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-model-cache\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.198798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.198665    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-home\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.198798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.198702    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tmp-dir\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.198798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.198767    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqrk4\" (UniqueName: \"kubernetes.io/projected/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-kube-api-access-zqrk4\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.199019 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.198875    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tls-certs\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.215747 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.206872    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"]
Oct 30 21:26:48.300022 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.299989    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqrk4\" (UniqueName: \"kubernetes.io/projected/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-kube-api-access-zqrk4\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.300230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.300104    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tls-certs\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.300230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.300152    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-dshm\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.300230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.300170    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-model-cache\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.300384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.300237    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-home\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.300384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.300277    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tmp-dir\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.300723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.300689    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-model-cache\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.300723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.300708    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tmp-dir\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.301048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.300759    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-home\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.302537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.302509    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-dshm\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.302824 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.302807    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tls-certs\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.360126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.360066    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqrk4\" (UniqueName: \"kubernetes.io/projected/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-kube-api-access-zqrk4\") pod \"simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") " pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.421147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.421121    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:48.799071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:48.799045    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"]
Oct 30 21:26:49.724876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:49.724837    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" event={"ID":"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5","Type":"ContainerStarted","Data":"5c606bd708026adbb62f1a914fdf52679c959dfdcfdfee93ba5fd3d584215511"}
Oct 30 21:26:49.724876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:49.724874    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" event={"ID":"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5","Type":"ContainerStarted","Data":"2ff4c754707caf0520a41832e8c01b153fc814b30faaeaec9fbea1361b78dab3"}
Oct 30 21:26:49.845815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:49.845762    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podStartSLOduration=1.845748427 podStartE2EDuration="1.845748427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:49.845165088 +0000 UTC m=+865.716284205" watchObservedRunningTime="2026-10-30 21:26:49.845748427 +0000 UTC m=+865.716867546"
Oct 30 21:26:50.421937 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:50.421913    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:52.353885 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:26:52.353852    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:26:58.421740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:58.421707    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:58.435491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:58.435468    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:26:58.788114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:26:58.788040    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:27:06.269294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.269261    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"]
Oct 30 21:27:06.272698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.272677    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.324915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.324889    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-be9451-openshift-default-dockercfg-lwh6g\""
Oct 30 21:27:06.371197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371169    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.371342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371318    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.371424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371402    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.371471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371446    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.371526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371507    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.371562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371537    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.371685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371667    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.371815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371786    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.371938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.371860    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhxp6\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-kube-api-access-hhxp6\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.376906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.376884    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"]
Oct 30 21:27:06.472808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.472780    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhxp6\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-kube-api-access-hhxp6\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.472955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.472866    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.472955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.472916    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.472955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.472946    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.472966    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.472987    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.473010    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.473109    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.473150    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.473513    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.473561    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.473597    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.473854    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.473933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.473914    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.475373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.475349    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.475469 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.475418    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.529183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.529129    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.529311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.529273    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhxp6\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-kube-api-access-hhxp6\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:06.585914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:06.585888    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:07.010775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:07.010755    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"]
Oct 30 21:27:07.012887 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:27:07.012859    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa02a459_0f58_4ccd_ba60_3b55c5d0e121.slice/crio-2d20244139a41b38b364881ed416fcc3efbfaf5872a2ca2e5af62d373b0543f7 WatchSource:0}: Error finding container 2d20244139a41b38b364881ed416fcc3efbfaf5872a2ca2e5af62d373b0543f7: Status 404 returned error can't find the container with id 2d20244139a41b38b364881ed416fcc3efbfaf5872a2ca2e5af62d373b0543f7
Oct 30 21:27:07.015028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:07.015002    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:27:07.015088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:07.015062    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:27:07.015123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:07.015090    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:27:07.842536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:07.842501    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" event={"ID":"aa02a459-0f58-4ccd-ba60-3b55c5d0e121","Type":"ContainerStarted","Data":"4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e"}
Oct 30 21:27:07.842908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:07.842550    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" event={"ID":"aa02a459-0f58-4ccd-ba60-3b55c5d0e121","Type":"ContainerStarted","Data":"2d20244139a41b38b364881ed416fcc3efbfaf5872a2ca2e5af62d373b0543f7"}
Oct 30 21:27:08.077741 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:08.077685    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" podStartSLOduration=2.077668566 podStartE2EDuration="2.077668566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:08.007508987 +0000 UTC m=+883.878628113" watchObservedRunningTime="2026-10-30 21:27:08.077668566 +0000 UTC m=+883.948787729"
Oct 30 21:27:08.586429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:08.586399    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:08.588080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:08.588048    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:27:08.588223 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:08.588093    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:27:09.586470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:09.586436    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:27:09.586880 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:09.586493    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:27:10.586443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:10.586408    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:27:10.586622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:10.586466    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:27:11.586461 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:11.586419    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:27:11.586672 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:11.586479    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:27:12.586199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:12.586166    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:27:12.586402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:12.586243    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:27:14.092024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:14.091989    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:14.092614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:14.092524    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:14.093873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:14.093839    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:27:22.353813 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:27:22.353774    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:27:34.805775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.805704    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"]
Oct 30 21:27:34.809812 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.809793    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:34.865698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.865677    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:27:34.891963 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.891941    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6krh\" (UniqueName: \"kubernetes.io/projected/b48e6537-d5eb-4097-9161-22777d9a919e-kube-api-access-f6krh\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:34.892065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.892046    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:34.892130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.892119    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:34.915548 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.915526    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:27:34.917701 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.917681    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"]
Oct 30 21:27:34.993350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.993330    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6krh\" (UniqueName: \"kubernetes.io/projected/b48e6537-d5eb-4097-9161-22777d9a919e-kube-api-access-f6krh\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:34.993443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.993382    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:34.993443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.993432    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:34.994087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.994069    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:34.995824 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:34.995806    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:35.049956 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:35.049929    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6krh\" (UniqueName: \"kubernetes.io/projected/b48e6537-d5eb-4097-9161-22777d9a919e-kube-api-access-f6krh\") pod \"maas-api-revoke-sub-9c44f461-glf2w\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:35.122734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:35.122683    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:35.496623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:35.496601    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"]
Oct 30 21:27:35.498791 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:27:35.498753    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb48e6537_d5eb_4097_9161_22777d9a919e.slice/crio-833a2fae8ca19f5947d1e9586fb46cf215a286d7a9669c9d3d22335570040eb8 WatchSource:0}: Error finding container 833a2fae8ca19f5947d1e9586fb46cf215a286d7a9669c9d3d22335570040eb8: Status 404 returned error can't find the container with id 833a2fae8ca19f5947d1e9586fb46cf215a286d7a9669c9d3d22335570040eb8
Oct 30 21:27:35.500599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:35.500582    5786 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:27:36.021100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:36.021069    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w" event={"ID":"b48e6537-d5eb-4097-9161-22777d9a919e","Type":"ContainerStarted","Data":"833a2fae8ca19f5947d1e9586fb46cf215a286d7a9669c9d3d22335570040eb8"}
Oct 30 21:27:41.053514 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:41.053485    5786 generic.go:356] "Generic (PLEG): container finished" podID="b48e6537-d5eb-4097-9161-22777d9a919e" containerID="a687bd023d0563ed11b6d749122157cb07e80f3d26217292c2b5596bedd90e49" exitCode=0
Oct 30 21:27:41.053928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:41.053579    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w" event={"ID":"b48e6537-d5eb-4097-9161-22777d9a919e","Type":"ContainerDied","Data":"a687bd023d0563ed11b6d749122157cb07e80f3d26217292c2b5596bedd90e49"}
Oct 30 21:27:42.208403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.208381    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:42.287987 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.287960    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b48e6537-d5eb-4097-9161-22777d9a919e-kube-api-access-f6krh\" (UniqueName: \"kubernetes.io/projected/b48e6537-d5eb-4097-9161-22777d9a919e-kube-api-access-f6krh\") pod \"b48e6537-d5eb-4097-9161-22777d9a919e\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") "
Oct 30 21:27:42.288105 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.288040    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-openshift-ca\") pod \"b48e6537-d5eb-4097-9161-22777d9a919e\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") "
Oct 30 21:27:42.288145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.288100    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-xks-ca\") pod \"b48e6537-d5eb-4097-9161-22777d9a919e\" (UID: \"b48e6537-d5eb-4097-9161-22777d9a919e\") "
Oct 30 21:27:42.288464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.288437    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-openshift-ca" pod "b48e6537-d5eb-4097-9161-22777d9a919e" (UID: "b48e6537-d5eb-4097-9161-22777d9a919e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:27:42.288561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.288535    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:27:42.290294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.290266    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-xks-ca" pod "b48e6537-d5eb-4097-9161-22777d9a919e" (UID: "b48e6537-d5eb-4097-9161-22777d9a919e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:27:42.290372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.290294    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b48e6537-d5eb-4097-9161-22777d9a919e-kube-api-access-f6krh" pod "b48e6537-d5eb-4097-9161-22777d9a919e" (UID: "b48e6537-d5eb-4097-9161-22777d9a919e"). InnerVolumeSpecName "kube-api-access-f6krh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:42.389621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.389585    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b48e6537-d5eb-4097-9161-22777d9a919e-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:27:42.389621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:42.389620    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6krh\" (UniqueName: \"kubernetes.io/projected/b48e6537-d5eb-4097-9161-22777d9a919e-kube-api-access-f6krh\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:27:43.065892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:43.065861    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w" event={"ID":"b48e6537-d5eb-4097-9161-22777d9a919e","Type":"ContainerDied","Data":"833a2fae8ca19f5947d1e9586fb46cf215a286d7a9669c9d3d22335570040eb8"}
Oct 30 21:27:43.066064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:43.065898    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"
Oct 30 21:27:43.066064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:27:43.065909    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="833a2fae8ca19f5947d1e9586fb46cf215a286d7a9669c9d3d22335570040eb8"
Oct 30 21:27:52.358080 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:27:52.358048    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:28:11.236068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.236032    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"]
Oct 30 21:28:11.236631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.236617    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b48e6537-d5eb-4097-9161-22777d9a919e" containerName="revoke-keys"
Oct 30 21:28:11.236673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.236634    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b48e6537-d5eb-4097-9161-22777d9a919e" containerName="revoke-keys"
Oct 30 21:28:11.236728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.236718    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b48e6537-d5eb-4097-9161-22777d9a919e" containerName="revoke-keys"
Oct 30 21:28:11.240185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.240168    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.292808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.292788    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:28:11.296084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.296063    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"]
Oct 30 21:28:11.310404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.310375    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.310546 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.310534    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95qvg\" (UniqueName: \"kubernetes.io/projected/be7f6acb-92fd-4e40-9124-15d6613a3cb5-kube-api-access-95qvg\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.310588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.310568    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.368698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.368663    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:28:11.411959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.411924    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95qvg\" (UniqueName: \"kubernetes.io/projected/be7f6acb-92fd-4e40-9124-15d6613a3cb5-kube-api-access-95qvg\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.412145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.411981    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.412145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.412033    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.412735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.412709    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.414380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.414356    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.468150 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.468127    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95qvg\" (UniqueName: \"kubernetes.io/projected/be7f6acb-92fd-4e40-9124-15d6613a3cb5-kube-api-access-95qvg\") pod \"maas-api-revoke-sub-a9e5ac9d-hghsn\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.553137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.553078    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:11.927934 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:11.927912    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"]
Oct 30 21:28:11.929808 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:28:11.929782    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7f6acb_92fd_4e40_9124_15d6613a3cb5.slice/crio-bf34b25fee7fd5d0b5fe3e00890d889bda6bd3d9b8621bfe9f4ef3f69c57c688 WatchSource:0}: Error finding container bf34b25fee7fd5d0b5fe3e00890d889bda6bd3d9b8621bfe9f4ef3f69c57c688: Status 404 returned error can't find the container with id bf34b25fee7fd5d0b5fe3e00890d889bda6bd3d9b8621bfe9f4ef3f69c57c688
Oct 30 21:28:12.238059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:12.238031    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn" event={"ID":"be7f6acb-92fd-4e40-9124-15d6613a3cb5","Type":"ContainerStarted","Data":"bf34b25fee7fd5d0b5fe3e00890d889bda6bd3d9b8621bfe9f4ef3f69c57c688"}
Oct 30 21:28:13.245027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:13.244996    5786 generic.go:356] "Generic (PLEG): container finished" podID="be7f6acb-92fd-4e40-9124-15d6613a3cb5" containerID="6e0810cd32da337fbb45fc135b5f170122ae65de9ead0af6241aac8fc2779d27" exitCode=0
Oct 30 21:28:13.245407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:13.245053    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn" event={"ID":"be7f6acb-92fd-4e40-9124-15d6613a3cb5","Type":"ContainerDied","Data":"6e0810cd32da337fbb45fc135b5f170122ae65de9ead0af6241aac8fc2779d27"}
Oct 30 21:28:14.423235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.420304    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:14.548192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.548050    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-xks-ca\") pod \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") "
Oct 30 21:28:14.548347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.548298    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-openshift-ca\") pod \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") "
Oct 30 21:28:14.548347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.548337    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be7f6acb-92fd-4e40-9124-15d6613a3cb5-kube-api-access-95qvg\" (UniqueName: \"kubernetes.io/projected/be7f6acb-92fd-4e40-9124-15d6613a3cb5-kube-api-access-95qvg\") pod \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\" (UID: \"be7f6acb-92fd-4e40-9124-15d6613a3cb5\") "
Oct 30 21:28:14.548651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.548628    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-openshift-ca" pod "be7f6acb-92fd-4e40-9124-15d6613a3cb5" (UID: "be7f6acb-92fd-4e40-9124-15d6613a3cb5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:14.548846 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.548829    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:14.550266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.550241    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-xks-ca" pod "be7f6acb-92fd-4e40-9124-15d6613a3cb5" (UID: "be7f6acb-92fd-4e40-9124-15d6613a3cb5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:28:14.550876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.550854    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be7f6acb-92fd-4e40-9124-15d6613a3cb5-kube-api-access-95qvg" pod "be7f6acb-92fd-4e40-9124-15d6613a3cb5" (UID: "be7f6acb-92fd-4e40-9124-15d6613a3cb5"). InnerVolumeSpecName "kube-api-access-95qvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:14.649893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.649869    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/be7f6acb-92fd-4e40-9124-15d6613a3cb5-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:14.650019 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:14.649896    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95qvg\" (UniqueName: \"kubernetes.io/projected/be7f6acb-92fd-4e40-9124-15d6613a3cb5-kube-api-access-95qvg\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:15.257944 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:15.257901    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn" event={"ID":"be7f6acb-92fd-4e40-9124-15d6613a3cb5","Type":"ContainerDied","Data":"bf34b25fee7fd5d0b5fe3e00890d889bda6bd3d9b8621bfe9f4ef3f69c57c688"}
Oct 30 21:28:15.257944 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:15.257926    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"
Oct 30 21:28:15.258139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:15.257957    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf34b25fee7fd5d0b5fe3e00890d889bda6bd3d9b8621bfe9f4ef3f69c57c688"
Oct 30 21:28:22.353698 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:28:22.353665    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:28:22.496070 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.496034    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"]
Oct 30 21:28:22.496662 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.496643    5786 cpu_manager.go:414] "containerMap: removing container" podUID="be7f6acb-92fd-4e40-9124-15d6613a3cb5" containerName="revoke-keys"
Oct 30 21:28:22.496765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.496663    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be7f6acb-92fd-4e40-9124-15d6613a3cb5" containerName="revoke-keys"
Oct 30 21:28:22.496852 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.496839    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="be7f6acb-92fd-4e40-9124-15d6613a3cb5" containerName="revoke-keys"
Oct 30 21:28:22.500322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.500304    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.552139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.552112    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-8ce31a-openshift-default-dockercfg-kn28f\""
Oct 30 21:28:22.604855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.604794    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"]
Oct 30 21:28:22.640258 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640234    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.640386 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640273    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.640442 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640389    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.640480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640441    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.640550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640533    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7765c1e6-c88f-4863-89ab-a19648eed81e-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.640598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640586    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.640639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640628    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbz76\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-kube-api-access-wbz76\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.640692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640659    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.640734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.640707    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.741831 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.741804    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7765c1e6-c88f-4863-89ab-a19648eed81e-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.741935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.741837    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.741935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.741873    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbz76\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-kube-api-access-wbz76\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.741935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.741893    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.741935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.741907    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.741974    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.741989    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.742020    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.742041    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.742318    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.742365    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.742448    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.742498    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.742562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.742530    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7765c1e6-c88f-4863-89ab-a19648eed81e-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.744700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.744674    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.744920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.744898    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.799326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.799302    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbz76\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-kube-api-access-wbz76\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.799556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.799538    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:22.814145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:22.814124    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:23.244627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:23.244600    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"]
Oct 30 21:28:23.246676 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:28:23.246654    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7765c1e6_c88f_4863_89ab_a19648eed81e.slice/crio-2d7511fb2ca7ea49f70310d9ae16f8b9aa46617fcc54740616ecf06f36500f58 WatchSource:0}: Error finding container 2d7511fb2ca7ea49f70310d9ae16f8b9aa46617fcc54740616ecf06f36500f58: Status 404 returned error can't find the container with id 2d7511fb2ca7ea49f70310d9ae16f8b9aa46617fcc54740616ecf06f36500f58
Oct 30 21:28:23.248827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:23.248795    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:28:23.248911 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:23.248861    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:28:23.248911 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:23.248893    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:28:23.306593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:23.306558    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" event={"ID":"7765c1e6-c88f-4863-89ab-a19648eed81e","Type":"ContainerStarted","Data":"2d7511fb2ca7ea49f70310d9ae16f8b9aa46617fcc54740616ecf06f36500f58"}
Oct 30 21:28:24.313630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:24.313598    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" event={"ID":"7765c1e6-c88f-4863-89ab-a19648eed81e","Type":"ContainerStarted","Data":"6e706c11b5e0218148e147526608cc74a3c4ba0780e54df398d5bebfc87adeef"}
Oct 30 21:28:24.548497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:24.548450    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" podStartSLOduration=2.548436219 podStartE2EDuration="2.548436219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:24.496932845 +0000 UTC m=+960.368051963" watchObservedRunningTime="2026-10-30 21:28:24.548436219 +0000 UTC m=+960.419555345"
Oct 30 21:28:24.814538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:24.814507    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:24.816303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:24.816274    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:28:24.816451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:24.816323    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:28:25.815125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:25.815090    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:28:25.815554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:25.815142    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:28:26.814439 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:26.814401    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:28:26.814616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:26.814454    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:28:27.814764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:27.814726    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:28:27.815274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:27.814781    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:28:28.815148 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:28.815111    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:28:28.815549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:28.815159    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:28:29.818531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:29.818504    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:29.819017 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:29.818897    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:29.819489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:29.819471    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:28:31.168441 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.168407    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"]
Oct 30 21:28:31.172469 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.172453    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.225616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.225588    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:28:31.261000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.260978    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-xks-ca\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.261137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.261113    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkst4\" (UniqueName: \"kubernetes.io/projected/c7b7589e-ffb2-46d5-92e4-181146eb0526-kube-api-access-hkst4\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.261253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.261239    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.274957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.274938    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:28:31.277289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.277270    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"]
Oct 30 21:28:31.362032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.362004    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.362147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.362118    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-xks-ca\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.362271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.362255    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkst4\" (UniqueName: \"kubernetes.io/projected/c7b7589e-ffb2-46d5-92e4-181146eb0526-kube-api-access-hkst4\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.362656 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.362639    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.364417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.364399    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-xks-ca\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.419268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.419190    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkst4\" (UniqueName: \"kubernetes.io/projected/c7b7589e-ffb2-46d5-92e4-181146eb0526-kube-api-access-hkst4\") pod \"maas-api-revoke-sub-35418c21-2vx6h\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.484576 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.484554    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:31.856674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:31.856652    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"]
Oct 30 21:28:31.858858 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:28:31.858830    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b7589e_ffb2_46d5_92e4_181146eb0526.slice/crio-773a12e8fd2b7c98319eaa3f03188b0f53a01a9460de92c49a090de9c15203d5 WatchSource:0}: Error finding container 773a12e8fd2b7c98319eaa3f03188b0f53a01a9460de92c49a090de9c15203d5: Status 404 returned error can't find the container with id 773a12e8fd2b7c98319eaa3f03188b0f53a01a9460de92c49a090de9c15203d5
Oct 30 21:28:32.359530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:32.359505    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h" event={"ID":"c7b7589e-ffb2-46d5-92e4-181146eb0526","Type":"ContainerStarted","Data":"e8fee44c20d0d8e70ccdde8438b87614ca4b37b97911f0728b1f3487fd25621c"}
Oct 30 21:28:32.359859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:32.359550    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h" event={"ID":"c7b7589e-ffb2-46d5-92e4-181146eb0526","Type":"ContainerStarted","Data":"773a12e8fd2b7c98319eaa3f03188b0f53a01a9460de92c49a090de9c15203d5"}
Oct 30 21:28:32.474733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:32.474684    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h" podStartSLOduration=1.474671502 podStartE2EDuration="1.474671502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:32.472323288 +0000 UTC m=+968.343442409" watchObservedRunningTime="2026-10-30 21:28:32.474671502 +0000 UTC m=+968.345790643"
Oct 30 21:28:33.366404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:33.366375    5786 generic.go:356] "Generic (PLEG): container finished" podID="c7b7589e-ffb2-46d5-92e4-181146eb0526" containerID="e8fee44c20d0d8e70ccdde8438b87614ca4b37b97911f0728b1f3487fd25621c" exitCode=0
Oct 30 21:28:33.366759 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:33.366461    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h" event={"ID":"c7b7589e-ffb2-46d5-92e4-181146eb0526","Type":"ContainerDied","Data":"e8fee44c20d0d8e70ccdde8438b87614ca4b37b97911f0728b1f3487fd25621c"}
Oct 30 21:28:34.645507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.645449    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:34.656696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.656666    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-xks-ca\") pod \"c7b7589e-ffb2-46d5-92e4-181146eb0526\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") "
Oct 30 21:28:34.656928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.656905    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-openshift-ca\") pod \"c7b7589e-ffb2-46d5-92e4-181146eb0526\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") "
Oct 30 21:28:34.657054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.656962    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7b7589e-ffb2-46d5-92e4-181146eb0526-kube-api-access-hkst4\" (UniqueName: \"kubernetes.io/projected/c7b7589e-ffb2-46d5-92e4-181146eb0526-kube-api-access-hkst4\") pod \"c7b7589e-ffb2-46d5-92e4-181146eb0526\" (UID: \"c7b7589e-ffb2-46d5-92e4-181146eb0526\") "
Oct 30 21:28:34.657551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.657406    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-openshift-ca" pod "c7b7589e-ffb2-46d5-92e4-181146eb0526" (UID: "c7b7589e-ffb2-46d5-92e4-181146eb0526"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:34.659913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.659886    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b7589e-ffb2-46d5-92e4-181146eb0526-kube-api-access-hkst4" pod "c7b7589e-ffb2-46d5-92e4-181146eb0526" (UID: "c7b7589e-ffb2-46d5-92e4-181146eb0526"). InnerVolumeSpecName "kube-api-access-hkst4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:34.663611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.663585    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-xks-ca" pod "c7b7589e-ffb2-46d5-92e4-181146eb0526" (UID: "c7b7589e-ffb2-46d5-92e4-181146eb0526"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:28:34.761231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.758271    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:34.761231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.758305    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkst4\" (UniqueName: \"kubernetes.io/projected/c7b7589e-ffb2-46d5-92e4-181146eb0526-kube-api-access-hkst4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:34.761231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:34.758324    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7b7589e-ffb2-46d5-92e4-181146eb0526-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:35.380282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:35.380247    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h" event={"ID":"c7b7589e-ffb2-46d5-92e4-181146eb0526","Type":"ContainerDied","Data":"773a12e8fd2b7c98319eaa3f03188b0f53a01a9460de92c49a090de9c15203d5"}
Oct 30 21:28:35.380282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:35.380279    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"
Oct 30 21:28:35.380282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:35.380291    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="773a12e8fd2b7c98319eaa3f03188b0f53a01a9460de92c49a090de9c15203d5"
Oct 30 21:28:39.350768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.350738    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"]
Oct 30 21:28:39.351380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.351364    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c7b7589e-ffb2-46d5-92e4-181146eb0526" containerName="revoke-keys"
Oct 30 21:28:39.351440 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.351382    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7b7589e-ffb2-46d5-92e4-181146eb0526" containerName="revoke-keys"
Oct 30 21:28:39.351508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.351499    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7b7589e-ffb2-46d5-92e4-181146eb0526" containerName="revoke-keys"
Oct 30 21:28:39.355251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.355232    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.411811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.411790    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"]
Oct 30 21:28:39.457748 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.457724    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:28:39.457884 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.457830    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:28:39.537089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.537059    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.537226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.537116    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.537226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.537191    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clc7n\" (UniqueName: \"kubernetes.io/projected/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-kube-api-access-clc7n\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.637849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.637785    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.637849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.637842    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.637996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.637877    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clc7n\" (UniqueName: \"kubernetes.io/projected/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-kube-api-access-clc7n\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.638629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.638573    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.640392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.640365    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.693959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.693935    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clc7n\" (UniqueName: \"kubernetes.io/projected/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-kube-api-access-clc7n\") pod \"maas-api-revoke-sub-111c5e3c-2lqdg\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:39.968082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:39.968054    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:40.340290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:40.340266    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"]
Oct 30 21:28:40.342493 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:28:40.342453    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee2c92f3_caad_4eb3_b403_c0d9d0eb9b2f.slice/crio-fd8ff61c238b37774c1f2de233a21c3a8e728d668cbf5292c26862ee32aa0708 WatchSource:0}: Error finding container fd8ff61c238b37774c1f2de233a21c3a8e728d668cbf5292c26862ee32aa0708: Status 404 returned error can't find the container with id fd8ff61c238b37774c1f2de233a21c3a8e728d668cbf5292c26862ee32aa0708
Oct 30 21:28:40.419709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:40.419685    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg" event={"ID":"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f","Type":"ContainerStarted","Data":"fd8ff61c238b37774c1f2de233a21c3a8e728d668cbf5292c26862ee32aa0708"}
Oct 30 21:28:41.427964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:41.427936    5786 generic.go:356] "Generic (PLEG): container finished" podID="ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f" containerID="6c9096be03f97b5a60cafa033ab70e9071aa1e34ad36d11bf3679ca179353113" exitCode=0
Oct 30 21:28:41.428329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:41.427999    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg" event={"ID":"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f","Type":"ContainerDied","Data":"6c9096be03f97b5a60cafa033ab70e9071aa1e34ad36d11bf3679ca179353113"}
Oct 30 21:28:42.565529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.565509    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:42.670918 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.670898    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-openshift-ca\") pod \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") "
Oct 30 21:28:42.671041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.671021    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-xks-ca\") pod \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") "
Oct 30 21:28:42.671081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.671041    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-kube-api-access-clc7n\" (UniqueName: \"kubernetes.io/projected/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-kube-api-access-clc7n\") pod \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\" (UID: \"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f\") "
Oct 30 21:28:42.671246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.671223    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-openshift-ca" pod "ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f" (UID: "ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:42.671348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.671337    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:42.673172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.673153    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-kube-api-access-clc7n" pod "ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f" (UID: "ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f"). InnerVolumeSpecName "kube-api-access-clc7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:42.673172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.673155    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-xks-ca" pod "ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f" (UID: "ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:28:42.773014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.772944    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:42.773014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:42.772979    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clc7n\" (UniqueName: \"kubernetes.io/projected/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f-kube-api-access-clc7n\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:43.420137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.420096    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"]
Oct 30 21:28:43.420731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.420711    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f" containerName="revoke-keys"
Oct 30 21:28:43.420840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.420733    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f" containerName="revoke-keys"
Oct 30 21:28:43.420896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.420880    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f" containerName="revoke-keys"
Oct 30 21:28:43.424414 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.424393    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.442641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.442617    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"
Oct 30 21:28:43.442744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.442642    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg" event={"ID":"ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f","Type":"ContainerDied","Data":"fd8ff61c238b37774c1f2de233a21c3a8e728d668cbf5292c26862ee32aa0708"}
Oct 30 21:28:43.442744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.442690    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd8ff61c238b37774c1f2de233a21c3a8e728d668cbf5292c26862ee32aa0708"
Oct 30 21:28:43.480126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.480015    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.480126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.480092    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g28zz\" (UniqueName: \"kubernetes.io/projected/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-kube-api-access-g28zz\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.480332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.480139    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.483709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.483686    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"]
Oct 30 21:28:43.581231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.581169    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.581649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.581271    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g28zz\" (UniqueName: \"kubernetes.io/projected/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-kube-api-access-g28zz\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.581649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.581315    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.581779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.581753    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.583778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.583757    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.638187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.638166    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g28zz\" (UniqueName: \"kubernetes.io/projected/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-kube-api-access-g28zz\") pod \"maas-api-revoke-sub-e4a80c0d-pv9dr\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:43.738379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:43.738330    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:44.110625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:44.110600    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"]
Oct 30 21:28:44.113701 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:28:44.113674    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe0ac47_cc07_4112_a3c3_c9a2884a6b83.slice/crio-63dbb1b04811796a60d7600632351e7d8603891c542a247d192cffb6d0c82979 WatchSource:0}: Error finding container 63dbb1b04811796a60d7600632351e7d8603891c542a247d192cffb6d0c82979: Status 404 returned error can't find the container with id 63dbb1b04811796a60d7600632351e7d8603891c542a247d192cffb6d0c82979
Oct 30 21:28:44.450298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:44.450268    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr" event={"ID":"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83","Type":"ContainerStarted","Data":"c7d8110844ac7a04e2ad671555f0a55d897c77aef5b301e2589da72e7fa91f88"}
Oct 30 21:28:44.450414 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:44.450316    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr" event={"ID":"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83","Type":"ContainerStarted","Data":"63dbb1b04811796a60d7600632351e7d8603891c542a247d192cffb6d0c82979"}
Oct 30 21:28:44.565100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:44.565057    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr" podStartSLOduration=1.565043636 podStartE2EDuration="1.565043636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:44.562251181 +0000 UTC m=+980.433370300" watchObservedRunningTime="2026-10-30 21:28:44.565043636 +0000 UTC m=+980.436162754"
Oct 30 21:28:45.457754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:45.457725    5786 generic.go:356] "Generic (PLEG): container finished" podID="3fe0ac47-cc07-4112-a3c3-c9a2884a6b83" containerID="c7d8110844ac7a04e2ad671555f0a55d897c77aef5b301e2589da72e7fa91f88" exitCode=0
Oct 30 21:28:45.458124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:45.457811    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr" event={"ID":"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83","Type":"ContainerDied","Data":"c7d8110844ac7a04e2ad671555f0a55d897c77aef5b301e2589da72e7fa91f88"}
Oct 30 21:28:46.605552 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.605525    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:46.713096 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.713071    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-xks-ca\") pod \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") "
Oct 30 21:28:46.713255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.713234    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-kube-api-access-g28zz\" (UniqueName: \"kubernetes.io/projected/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-kube-api-access-g28zz\") pod \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") "
Oct 30 21:28:46.713336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.713319    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-openshift-ca\") pod \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\" (UID: \"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83\") "
Oct 30 21:28:46.713672 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.713618    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-openshift-ca" pod "3fe0ac47-cc07-4112-a3c3-c9a2884a6b83" (UID: "3fe0ac47-cc07-4112-a3c3-c9a2884a6b83"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:46.713786 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.713740    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:46.715185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.715165    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-xks-ca" pod "3fe0ac47-cc07-4112-a3c3-c9a2884a6b83" (UID: "3fe0ac47-cc07-4112-a3c3-c9a2884a6b83"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:28:46.715336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.715317    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-kube-api-access-g28zz" pod "3fe0ac47-cc07-4112-a3c3-c9a2884a6b83" (UID: "3fe0ac47-cc07-4112-a3c3-c9a2884a6b83"). InnerVolumeSpecName "kube-api-access-g28zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:46.814932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.814905    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:46.814932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:46.814930    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g28zz\" (UniqueName: \"kubernetes.io/projected/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83-kube-api-access-g28zz\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:47.479578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:47.479551    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"
Oct 30 21:28:47.479757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:47.479580    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr" event={"ID":"3fe0ac47-cc07-4112-a3c3-c9a2884a6b83","Type":"ContainerDied","Data":"63dbb1b04811796a60d7600632351e7d8603891c542a247d192cffb6d0c82979"}
Oct 30 21:28:47.479757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:47.479662    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63dbb1b04811796a60d7600632351e7d8603891c542a247d192cffb6d0c82979"
Oct 30 21:28:52.354768 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:28:52.354731    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:28:55.630057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.630030    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"]
Oct 30 21:28:55.630561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.630546    5786 cpu_manager.go:414] "containerMap: removing container" podUID="3fe0ac47-cc07-4112-a3c3-c9a2884a6b83" containerName="revoke-keys"
Oct 30 21:28:55.630561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.630561    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fe0ac47-cc07-4112-a3c3-c9a2884a6b83" containerName="revoke-keys"
Oct 30 21:28:55.630691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.630683    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fe0ac47-cc07-4112-a3c3-c9a2884a6b83" containerName="revoke-keys"
Oct 30 21:28:55.637280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.636070    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.690063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.690041    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"]
Oct 30 21:28:55.700803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.700778    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.700936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.700916    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.701011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.700952    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gwbf\" (UniqueName: \"kubernetes.io/projected/a9fcf672-636f-4587-b829-c0989b4018fc-kube-api-access-2gwbf\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.736534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.736514    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:28:55.736627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.736542    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:28:55.801665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.801643    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.801763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.801675    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gwbf\" (UniqueName: \"kubernetes.io/projected/a9fcf672-636f-4587-b829-c0989b4018fc-kube-api-access-2gwbf\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.801763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.801728    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.802351 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.802332    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.804147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.804131    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.857853 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.857823    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gwbf\" (UniqueName: \"kubernetes.io/projected/a9fcf672-636f-4587-b829-c0989b4018fc-kube-api-access-2gwbf\") pod \"maas-api-revoke-sub-06ae89ef-rtbxq\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:55.954680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:55.954659    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:56.328050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:56.328031    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"]
Oct 30 21:28:56.330190 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:28:56.330161    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fcf672_636f_4587_b829_c0989b4018fc.slice/crio-af0fba2c1f52a73ced520ea549902ede98cb3051725284ad2fa20ce499bf8d21 WatchSource:0}: Error finding container af0fba2c1f52a73ced520ea549902ede98cb3051725284ad2fa20ce499bf8d21: Status 404 returned error can't find the container with id af0fba2c1f52a73ced520ea549902ede98cb3051725284ad2fa20ce499bf8d21
Oct 30 21:28:56.531589 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:56.531557    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq" event={"ID":"a9fcf672-636f-4587-b829-c0989b4018fc","Type":"ContainerStarted","Data":"af0fba2c1f52a73ced520ea549902ede98cb3051725284ad2fa20ce499bf8d21"}
Oct 30 21:28:57.539077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:57.538997    5786 generic.go:356] "Generic (PLEG): container finished" podID="a9fcf672-636f-4587-b829-c0989b4018fc" containerID="dc5210ab2b12fe007e0230b4d1e4029269934b0d43db031770349226d65c77ec" exitCode=0
Oct 30 21:28:57.539459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:57.539078    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq" event={"ID":"a9fcf672-636f-4587-b829-c0989b4018fc","Type":"ContainerDied","Data":"dc5210ab2b12fe007e0230b4d1e4029269934b0d43db031770349226d65c77ec"}
Oct 30 21:28:58.720754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.720727    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:58.841746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.841677    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-xks-ca\") pod \"a9fcf672-636f-4587-b829-c0989b4018fc\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") "
Oct 30 21:28:58.841746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.841728    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9fcf672-636f-4587-b829-c0989b4018fc-kube-api-access-2gwbf\" (UniqueName: \"kubernetes.io/projected/a9fcf672-636f-4587-b829-c0989b4018fc-kube-api-access-2gwbf\") pod \"a9fcf672-636f-4587-b829-c0989b4018fc\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") "
Oct 30 21:28:58.841950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.841822    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-openshift-ca\") pod \"a9fcf672-636f-4587-b829-c0989b4018fc\" (UID: \"a9fcf672-636f-4587-b829-c0989b4018fc\") "
Oct 30 21:28:58.842172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.842142    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-openshift-ca" pod "a9fcf672-636f-4587-b829-c0989b4018fc" (UID: "a9fcf672-636f-4587-b829-c0989b4018fc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:58.843995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.843967    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9fcf672-636f-4587-b829-c0989b4018fc-kube-api-access-2gwbf" pod "a9fcf672-636f-4587-b829-c0989b4018fc" (UID: "a9fcf672-636f-4587-b829-c0989b4018fc"). InnerVolumeSpecName "kube-api-access-2gwbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:58.844268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.844246    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-xks-ca" pod "a9fcf672-636f-4587-b829-c0989b4018fc" (UID: "a9fcf672-636f-4587-b829-c0989b4018fc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:28:58.956280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.949196    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:58.956280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.949251    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9fcf672-636f-4587-b829-c0989b4018fc-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:58.956280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:58.949266    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gwbf\" (UniqueName: \"kubernetes.io/projected/a9fcf672-636f-4587-b829-c0989b4018fc-kube-api-access-2gwbf\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:28:59.551286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:59.551249    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"
Oct 30 21:28:59.551455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:59.551248    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq" event={"ID":"a9fcf672-636f-4587-b829-c0989b4018fc","Type":"ContainerDied","Data":"af0fba2c1f52a73ced520ea549902ede98cb3051725284ad2fa20ce499bf8d21"}
Oct 30 21:28:59.551455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:28:59.551370    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af0fba2c1f52a73ced520ea549902ede98cb3051725284ad2fa20ce499bf8d21"
Oct 30 21:29:02.567890 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.567858    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"]
Oct 30 21:29:02.568540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.568522    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a9fcf672-636f-4587-b829-c0989b4018fc" containerName="revoke-keys"
Oct 30 21:29:02.568630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.568544    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9fcf672-636f-4587-b829-c0989b4018fc" containerName="revoke-keys"
Oct 30 21:29:02.568630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.568625    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9fcf672-636f-4587-b829-c0989b4018fc" containerName="revoke-keys"
Oct 30 21:29:02.572061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.572046    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.626032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.626003    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:29:02.675272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.675246    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:29:02.679416 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.679393    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"]
Oct 30 21:29:02.685580 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.685557    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slh7f\" (UniqueName: \"kubernetes.io/projected/cedb2623-365a-4fa1-b904-657fc204b8ea-kube-api-access-slh7f\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.685747 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.685730    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.685794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.685763    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.786658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.786629    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.786754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.786670    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.786821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.786761    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slh7f\" (UniqueName: \"kubernetes.io/projected/cedb2623-365a-4fa1-b904-657fc204b8ea-kube-api-access-slh7f\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.787332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.787309    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.789051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.789030    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.843809 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.843755    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slh7f\" (UniqueName: \"kubernetes.io/projected/cedb2623-365a-4fa1-b904-657fc204b8ea-kube-api-access-slh7f\") pod \"maas-api-revoke-sub-055ecf74-snllc\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:02.884249 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:02.884231    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:03.256460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:03.256436    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"]
Oct 30 21:29:03.258613 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:03.258588    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedb2623_365a_4fa1_b904_657fc204b8ea.slice/crio-0e8615fcbfd032fdab19435be396265c0c10f3b3706cddeffb6ceb4ab60e3b58 WatchSource:0}: Error finding container 0e8615fcbfd032fdab19435be396265c0c10f3b3706cddeffb6ceb4ab60e3b58: Status 404 returned error can't find the container with id 0e8615fcbfd032fdab19435be396265c0c10f3b3706cddeffb6ceb4ab60e3b58
Oct 30 21:29:03.576728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:03.576701    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc" event={"ID":"cedb2623-365a-4fa1-b904-657fc204b8ea","Type":"ContainerStarted","Data":"b6b7f7f58136d5f7bb83256ca014fe62ec9eaf799273929b1001d3188dc98204"}
Oct 30 21:29:03.577082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:03.576747    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc" event={"ID":"cedb2623-365a-4fa1-b904-657fc204b8ea","Type":"ContainerStarted","Data":"0e8615fcbfd032fdab19435be396265c0c10f3b3706cddeffb6ceb4ab60e3b58"}
Oct 30 21:29:03.695155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:03.695096    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc" podStartSLOduration=1.69507773 podStartE2EDuration="1.69507773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:29:03.692648867 +0000 UTC m=+999.563767986" watchObservedRunningTime="2026-10-30 21:29:03.69507773 +0000 UTC m=+999.566196852"
Oct 30 21:29:04.584005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:04.583978    5786 generic.go:356] "Generic (PLEG): container finished" podID="cedb2623-365a-4fa1-b904-657fc204b8ea" containerID="b6b7f7f58136d5f7bb83256ca014fe62ec9eaf799273929b1001d3188dc98204" exitCode=0
Oct 30 21:29:04.584418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:04.584069    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc" event={"ID":"cedb2623-365a-4fa1-b904-657fc204b8ea","Type":"ContainerDied","Data":"b6b7f7f58136d5f7bb83256ca014fe62ec9eaf799273929b1001d3188dc98204"}
Oct 30 21:29:05.733077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.733053    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:05.818087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.818063    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cedb2623-365a-4fa1-b904-657fc204b8ea-kube-api-access-slh7f\" (UniqueName: \"kubernetes.io/projected/cedb2623-365a-4fa1-b904-657fc204b8ea-kube-api-access-slh7f\") pod \"cedb2623-365a-4fa1-b904-657fc204b8ea\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") "
Oct 30 21:29:05.818262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.818246    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-xks-ca\") pod \"cedb2623-365a-4fa1-b904-657fc204b8ea\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") "
Oct 30 21:29:05.818368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.818354    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-openshift-ca\") pod \"cedb2623-365a-4fa1-b904-657fc204b8ea\" (UID: \"cedb2623-365a-4fa1-b904-657fc204b8ea\") "
Oct 30 21:29:05.818636 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.818613    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-openshift-ca" pod "cedb2623-365a-4fa1-b904-657fc204b8ea" (UID: "cedb2623-365a-4fa1-b904-657fc204b8ea"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:05.818746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.818732    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:05.820163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.820145    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cedb2623-365a-4fa1-b904-657fc204b8ea-kube-api-access-slh7f" pod "cedb2623-365a-4fa1-b904-657fc204b8ea" (UID: "cedb2623-365a-4fa1-b904-657fc204b8ea"). InnerVolumeSpecName "kube-api-access-slh7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:05.820306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.820288    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-xks-ca" pod "cedb2623-365a-4fa1-b904-657fc204b8ea" (UID: "cedb2623-365a-4fa1-b904-657fc204b8ea"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:05.919751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.919721    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedb2623-365a-4fa1-b904-657fc204b8ea-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:05.919751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:05.919748    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slh7f\" (UniqueName: \"kubernetes.io/projected/cedb2623-365a-4fa1-b904-657fc204b8ea-kube-api-access-slh7f\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:06.549094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.549063    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"]
Oct 30 21:29:06.549662 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.549648    5786 cpu_manager.go:414] "containerMap: removing container" podUID="cedb2623-365a-4fa1-b904-657fc204b8ea" containerName="revoke-keys"
Oct 30 21:29:06.549718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.549664    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cedb2623-365a-4fa1-b904-657fc204b8ea" containerName="revoke-keys"
Oct 30 21:29:06.549778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.549770    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="cedb2623-365a-4fa1-b904-657fc204b8ea" containerName="revoke-keys"
Oct 30 21:29:06.553191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.553171    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.597089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.597064    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc" event={"ID":"cedb2623-365a-4fa1-b904-657fc204b8ea","Type":"ContainerDied","Data":"0e8615fcbfd032fdab19435be396265c0c10f3b3706cddeffb6ceb4ab60e3b58"}
Oct 30 21:29:06.597178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.597096    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"
Oct 30 21:29:06.597178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.597107    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e8615fcbfd032fdab19435be396265c0c10f3b3706cddeffb6ceb4ab60e3b58"
Oct 30 21:29:06.605080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.605064    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-9f52c3-openshift-default-dockercfg-ftmnt\""
Oct 30 21:29:06.626754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.626688    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.626832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.626806    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.626872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.626830    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.626906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.626878    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.626939 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.626915    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.627010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.626992    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f882f-d0de-48b8-839d-e33a4fd911fe-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.627047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.627038    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.627099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.627084    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ndqc\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-kube-api-access-9ndqc\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.627133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.627115    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.657614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.657593    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"]
Oct 30 21:29:06.728618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728593    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.728749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728664    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.728749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728680    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.728749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728708    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.728749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728732    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.728955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728756    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f882f-d0de-48b8-839d-e33a4fd911fe-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.728955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728850    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.728955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728905    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ndqc\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-kube-api-access-9ndqc\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.728955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.728937    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.729155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.729036    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.729252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.729173    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.729313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.729265    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.729367 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.729324    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.729510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.729472    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f882f-d0de-48b8-839d-e33a4fd911fe-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.731034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.731013    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.731246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.731230    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.785306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.785284    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.785601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.785572    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ndqc\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-kube-api-access-9ndqc\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:06.866627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:06.866604    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:07.294585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:07.294561    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"]
Oct 30 21:29:07.296303 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:07.296271    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969f882f_d0de_48b8_839d_e33a4fd911fe.slice/crio-1cc2f21918e7053de06961f63a354e6c47970dbb7456fe5e1c1c2f3a8e30193b WatchSource:0}: Error finding container 1cc2f21918e7053de06961f63a354e6c47970dbb7456fe5e1c1c2f3a8e30193b: Status 404 returned error can't find the container with id 1cc2f21918e7053de06961f63a354e6c47970dbb7456fe5e1c1c2f3a8e30193b
Oct 30 21:29:07.298670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:07.298641    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:29:07.298745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:07.298704    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:29:07.298745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:07.298729    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:29:07.604940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:07.604844    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" event={"ID":"969f882f-d0de-48b8-839d-e33a4fd911fe","Type":"ContainerStarted","Data":"8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6"}
Oct 30 21:29:07.604940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:07.604896    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" event={"ID":"969f882f-d0de-48b8-839d-e33a4fd911fe","Type":"ContainerStarted","Data":"1cc2f21918e7053de06961f63a354e6c47970dbb7456fe5e1c1c2f3a8e30193b"}
Oct 30 21:29:07.723231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:07.723158    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" podStartSLOduration=1.723142909 podStartE2EDuration="1.723142909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:29:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:29:07.721943645 +0000 UTC m=+1003.593062764" watchObservedRunningTime="2026-10-30 21:29:07.723142909 +0000 UTC m=+1003.594262028"
Oct 30 21:29:07.874286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:07.867005    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:08.876338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:08.876299    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:29:08.876693 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:08.876360    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:29:09.873428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:09.872837    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:29:09.873428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:09.872898    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:29:10.493235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.493192    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"]
Oct 30 21:29:10.497342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.497323    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.522027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.522004    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.522135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.522043    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.522223 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.522191    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6czmp\" (UniqueName: \"kubernetes.io/projected/b6d7de61-b323-40c5-8b7b-c0726b871be0-kube-api-access-6czmp\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.549879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.549859    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:29:10.550008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.549859    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:29:10.578648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.578618    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"]
Oct 30 21:29:10.623480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.623448    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.623618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.623495    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.623618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.623606    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6czmp\" (UniqueName: \"kubernetes.io/projected/b6d7de61-b323-40c5-8b7b-c0726b871be0-kube-api-access-6czmp\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.624048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.624025    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.625960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.625932    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.680889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.680859    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6czmp\" (UniqueName: \"kubernetes.io/projected/b6d7de61-b323-40c5-8b7b-c0726b871be0-kube-api-access-6czmp\") pod \"maas-api-revoke-sub-31ba00c4-wtd54\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.811446 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.811382    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:10.876231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.875571    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:29:10.876231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:10.875645    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:29:11.211403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:11.211379    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"]
Oct 30 21:29:11.213355 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:11.213320    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d7de61_b323_40c5_8b7b_c0726b871be0.slice/crio-6ca32feb903317ad4b4f2e8cb4d9c04b38b34f7e202ea730305f2363e0b8bf84 WatchSource:0}: Error finding container 6ca32feb903317ad4b4f2e8cb4d9c04b38b34f7e202ea730305f2363e0b8bf84: Status 404 returned error can't find the container with id 6ca32feb903317ad4b4f2e8cb4d9c04b38b34f7e202ea730305f2363e0b8bf84
Oct 30 21:29:11.631092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:11.631060    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54" event={"ID":"b6d7de61-b323-40c5-8b7b-c0726b871be0","Type":"ContainerStarted","Data":"91ee1d748940cd5344e2261615e0cb28a251c4d9f08814016a510e8d286dabbd"}
Oct 30 21:29:11.631411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:11.631110    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54" event={"ID":"b6d7de61-b323-40c5-8b7b-c0726b871be0","Type":"ContainerStarted","Data":"6ca32feb903317ad4b4f2e8cb4d9c04b38b34f7e202ea730305f2363e0b8bf84"}
Oct 30 21:29:11.744403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:11.744355    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54" podStartSLOduration=1.744341905 podStartE2EDuration="1.744341905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:29:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:29:11.743371946 +0000 UTC m=+1007.614491065" watchObservedRunningTime="2026-10-30 21:29:11.744341905 +0000 UTC m=+1007.615461024"
Oct 30 21:29:11.872290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:11.872183    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:29:11.872511 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:11.872482    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:29:12.638005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:12.637919    5786 generic.go:356] "Generic (PLEG): container finished" podID="b6d7de61-b323-40c5-8b7b-c0726b871be0" containerID="91ee1d748940cd5344e2261615e0cb28a251c4d9f08814016a510e8d286dabbd" exitCode=0
Oct 30 21:29:12.638369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:12.638007    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54" event={"ID":"b6d7de61-b323-40c5-8b7b-c0726b871be0","Type":"ContainerDied","Data":"91ee1d748940cd5344e2261615e0cb28a251c4d9f08814016a510e8d286dabbd"}
Oct 30 21:29:12.873615 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:12.873579    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:29:12.873873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:12.873846    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:29:13.794169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.794145    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:13.866982 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.866962    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6d7de61-b323-40c5-8b7b-c0726b871be0-kube-api-access-6czmp\" (UniqueName: \"kubernetes.io/projected/b6d7de61-b323-40c5-8b7b-c0726b871be0-kube-api-access-6czmp\") pod \"b6d7de61-b323-40c5-8b7b-c0726b871be0\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") "
Oct 30 21:29:13.867111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.867022    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-xks-ca\") pod \"b6d7de61-b323-40c5-8b7b-c0726b871be0\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") "
Oct 30 21:29:13.867111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.867101    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-openshift-ca\") pod \"b6d7de61-b323-40c5-8b7b-c0726b871be0\" (UID: \"b6d7de61-b323-40c5-8b7b-c0726b871be0\") "
Oct 30 21:29:13.867485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.867457    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-openshift-ca" pod "b6d7de61-b323-40c5-8b7b-c0726b871be0" (UID: "b6d7de61-b323-40c5-8b7b-c0726b871be0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:13.867793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.867527    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.869289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.869265    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d7de61-b323-40c5-8b7b-c0726b871be0-kube-api-access-6czmp" pod "b6d7de61-b323-40c5-8b7b-c0726b871be0" (UID: "b6d7de61-b323-40c5-8b7b-c0726b871be0"). InnerVolumeSpecName "kube-api-access-6czmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:13.869373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.869312    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-xks-ca" pod "b6d7de61-b323-40c5-8b7b-c0726b871be0" (UID: "b6d7de61-b323-40c5-8b7b-c0726b871be0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:13.871899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.871881    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:13.872234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.872218    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:13.872902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.872881    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:29:13.897259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.897193    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"]
Oct 30 21:29:13.897803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.897785    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b6d7de61-b323-40c5-8b7b-c0726b871be0" containerName="revoke-keys"
Oct 30 21:29:13.897803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.897802    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6d7de61-b323-40c5-8b7b-c0726b871be0" containerName="revoke-keys"
Oct 30 21:29:13.897964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.897943    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6d7de61-b323-40c5-8b7b-c0726b871be0" containerName="revoke-keys"
Oct 30 21:29:13.901344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.901329    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:13.968401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.968379    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:13.968633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.968608    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:13.968706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.968672    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm5mz\" (UniqueName: \"kubernetes.io/projected/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-kube-api-access-bm5mz\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:13.968768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.968754    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6czmp\" (UniqueName: \"kubernetes.io/projected/b6d7de61-b323-40c5-8b7b-c0726b871be0-kube-api-access-6czmp\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.968821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.968779    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b6d7de61-b323-40c5-8b7b-c0726b871be0-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.990925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:13.990906    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"]
Oct 30 21:29:14.069605 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.069571    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:14.069771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.069753    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:14.069834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.069820    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm5mz\" (UniqueName: \"kubernetes.io/projected/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-kube-api-access-bm5mz\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:14.070288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.070267    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:14.072301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.072274    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:14.126691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.126672    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm5mz\" (UniqueName: \"kubernetes.io/projected/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-kube-api-access-bm5mz\") pod \"maas-api-revoke-sub-b67865fd-75sgr\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:14.213886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.213863    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:14.589521 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.589495    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"]
Oct 30 21:29:14.591552 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:14.591528    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf67ccb_ccb7_4691_a39f_6d2191eeae74.slice/crio-d389553aecbe3996beb400033fd767798f82371ba1ca87218fab530760f0f2fd WatchSource:0}: Error finding container d389553aecbe3996beb400033fd767798f82371ba1ca87218fab530760f0f2fd: Status 404 returned error can't find the container with id d389553aecbe3996beb400033fd767798f82371ba1ca87218fab530760f0f2fd
Oct 30 21:29:14.653086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.653066    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"
Oct 30 21:29:14.653182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.653066    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54" event={"ID":"b6d7de61-b323-40c5-8b7b-c0726b871be0","Type":"ContainerDied","Data":"6ca32feb903317ad4b4f2e8cb4d9c04b38b34f7e202ea730305f2363e0b8bf84"}
Oct 30 21:29:14.653260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.653188    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ca32feb903317ad4b4f2e8cb4d9c04b38b34f7e202ea730305f2363e0b8bf84"
Oct 30 21:29:14.654667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:14.654644    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr" event={"ID":"ccf67ccb-ccb7-4691-a39f-6d2191eeae74","Type":"ContainerStarted","Data":"d389553aecbe3996beb400033fd767798f82371ba1ca87218fab530760f0f2fd"}
Oct 30 21:29:15.106532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.106503    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"]
Oct 30 21:29:15.111142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.111088    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.168822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.168795    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"]
Oct 30 21:29:15.182125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.182099    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.182346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.182329    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.182395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.182355    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6klnq\" (UniqueName: \"kubernetes.io/projected/deba8191-7edf-47a7-8218-390d486e5e15-kube-api-access-6klnq\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.283449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.283426    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.283574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.283454    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6klnq\" (UniqueName: \"kubernetes.io/projected/deba8191-7edf-47a7-8218-390d486e5e15-kube-api-access-6klnq\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.283574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.283524    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.284168 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.284145    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.285764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.285749    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.340069 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.340045    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6klnq\" (UniqueName: \"kubernetes.io/projected/deba8191-7edf-47a7-8218-390d486e5e15-kube-api-access-6klnq\") pod \"maas-api-revoke-sub-a00c2a6f-4nzw5\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.424263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.424241    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:15.661574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.661537    5786 generic.go:356] "Generic (PLEG): container finished" podID="ccf67ccb-ccb7-4691-a39f-6d2191eeae74" containerID="d89a4b3d5968969a2807ecc3df72dbb289948222c39a93ac406b4f59c264ae8a" exitCode=0
Oct 30 21:29:15.661726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.661607    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr" event={"ID":"ccf67ccb-ccb7-4691-a39f-6d2191eeae74","Type":"ContainerDied","Data":"d89a4b3d5968969a2807ecc3df72dbb289948222c39a93ac406b4f59c264ae8a"}
Oct 30 21:29:15.797344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:15.797314    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"]
Oct 30 21:29:16.668841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.668815    5786 generic.go:356] "Generic (PLEG): container finished" podID="deba8191-7edf-47a7-8218-390d486e5e15" containerID="4d333f0e9e387ffe9727fb7c01f4bee62c779719ee3524349ddece2837067929" exitCode=0
Oct 30 21:29:16.669235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.668857    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5" event={"ID":"deba8191-7edf-47a7-8218-390d486e5e15","Type":"ContainerDied","Data":"4d333f0e9e387ffe9727fb7c01f4bee62c779719ee3524349ddece2837067929"}
Oct 30 21:29:16.669235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.668909    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5" event={"ID":"deba8191-7edf-47a7-8218-390d486e5e15","Type":"ContainerStarted","Data":"8ab478f9654a88b685fe8fe2bd4732cf2fd36e4bb7c3acbbe1b9b994bc31132f"}
Oct 30 21:29:16.824744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.824721    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:16.916609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.916584    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-openshift-ca\") pod \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") "
Oct 30 21:29:16.916755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.916706    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-kube-api-access-bm5mz\" (UniqueName: \"kubernetes.io/projected/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-kube-api-access-bm5mz\") pod \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") "
Oct 30 21:29:16.916755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.916747    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-xks-ca\") pod \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\" (UID: \"ccf67ccb-ccb7-4691-a39f-6d2191eeae74\") "
Oct 30 21:29:16.916901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.916881    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-openshift-ca" pod "ccf67ccb-ccb7-4691-a39f-6d2191eeae74" (UID: "ccf67ccb-ccb7-4691-a39f-6d2191eeae74"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:16.917194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.917178    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:16.918807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.918781    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-kube-api-access-bm5mz" pod "ccf67ccb-ccb7-4691-a39f-6d2191eeae74" (UID: "ccf67ccb-ccb7-4691-a39f-6d2191eeae74"). InnerVolumeSpecName "kube-api-access-bm5mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:16.918903 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:16.918819    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-xks-ca" pod "ccf67ccb-ccb7-4691-a39f-6d2191eeae74" (UID: "ccf67ccb-ccb7-4691-a39f-6d2191eeae74"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:17.018318 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:17.018289    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm5mz\" (UniqueName: \"kubernetes.io/projected/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-kube-api-access-bm5mz\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:17.018318 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:17.018314    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccf67ccb-ccb7-4691-a39f-6d2191eeae74-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:17.676573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:17.676542    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"
Oct 30 21:29:17.676573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:17.676562    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr" event={"ID":"ccf67ccb-ccb7-4691-a39f-6d2191eeae74","Type":"ContainerDied","Data":"d389553aecbe3996beb400033fd767798f82371ba1ca87218fab530760f0f2fd"}
Oct 30 21:29:17.677004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:17.676607    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d389553aecbe3996beb400033fd767798f82371ba1ca87218fab530760f0f2fd"
Oct 30 21:29:17.872928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:17.872908    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:18.032384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.032309    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-xks-ca\") pod \"deba8191-7edf-47a7-8218-390d486e5e15\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") "
Oct 30 21:29:18.032384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.032349    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deba8191-7edf-47a7-8218-390d486e5e15-kube-api-access-6klnq\" (UniqueName: \"kubernetes.io/projected/deba8191-7edf-47a7-8218-390d486e5e15-kube-api-access-6klnq\") pod \"deba8191-7edf-47a7-8218-390d486e5e15\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") "
Oct 30 21:29:18.032601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.032484    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-openshift-ca\") pod \"deba8191-7edf-47a7-8218-390d486e5e15\" (UID: \"deba8191-7edf-47a7-8218-390d486e5e15\") "
Oct 30 21:29:18.032845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.032816    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-openshift-ca" pod "deba8191-7edf-47a7-8218-390d486e5e15" (UID: "deba8191-7edf-47a7-8218-390d486e5e15"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:18.032985 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.032969    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:18.034632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.034607    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-xks-ca" pod "deba8191-7edf-47a7-8218-390d486e5e15" (UID: "deba8191-7edf-47a7-8218-390d486e5e15"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:18.034890 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.034874    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deba8191-7edf-47a7-8218-390d486e5e15-kube-api-access-6klnq" pod "deba8191-7edf-47a7-8218-390d486e5e15" (UID: "deba8191-7edf-47a7-8218-390d486e5e15"). InnerVolumeSpecName "kube-api-access-6klnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:18.134368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.134346    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/deba8191-7edf-47a7-8218-390d486e5e15-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:18.134368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.134367    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6klnq\" (UniqueName: \"kubernetes.io/projected/deba8191-7edf-47a7-8218-390d486e5e15-kube-api-access-6klnq\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:18.735230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.732090    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5" event={"ID":"deba8191-7edf-47a7-8218-390d486e5e15","Type":"ContainerDied","Data":"8ab478f9654a88b685fe8fe2bd4732cf2fd36e4bb7c3acbbe1b9b994bc31132f"}
Oct 30 21:29:18.735230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.732194    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ab478f9654a88b685fe8fe2bd4732cf2fd36e4bb7c3acbbe1b9b994bc31132f"
Oct 30 21:29:18.735230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:18.732377    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"
Oct 30 21:29:21.217254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.217219    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"]
Oct 30 21:29:21.217825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.217810    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ccf67ccb-ccb7-4691-a39f-6d2191eeae74" containerName="revoke-keys"
Oct 30 21:29:21.217886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.217826    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ccf67ccb-ccb7-4691-a39f-6d2191eeae74" containerName="revoke-keys"
Oct 30 21:29:21.217886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.217845    5786 cpu_manager.go:414] "containerMap: removing container" podUID="deba8191-7edf-47a7-8218-390d486e5e15" containerName="revoke-keys"
Oct 30 21:29:21.217886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.217849    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deba8191-7edf-47a7-8218-390d486e5e15" containerName="revoke-keys"
Oct 30 21:29:21.217979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.217953    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ccf67ccb-ccb7-4691-a39f-6d2191eeae74" containerName="revoke-keys"
Oct 30 21:29:21.217979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.217963    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="deba8191-7edf-47a7-8218-390d486e5e15" containerName="revoke-keys"
Oct 30 21:29:21.221499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.221486    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.273226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.273185    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:29:21.273352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.273337    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:29:21.305166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.305143    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mj9g\" (UniqueName: \"kubernetes.io/projected/e7a24106-afed-4372-ab3b-d9ea2773bf63-kube-api-access-6mj9g\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.305339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.305192    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.308302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.305406    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.308302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.305531    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"]
Oct 30 21:29:21.406557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.406533    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.406687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.406642    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.406791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.406776    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mj9g\" (UniqueName: \"kubernetes.io/projected/e7a24106-afed-4372-ab3b-d9ea2773bf63-kube-api-access-6mj9g\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.407242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.407192    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.408951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.408929    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.463820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.463797    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mj9g\" (UniqueName: \"kubernetes.io/projected/e7a24106-afed-4372-ab3b-d9ea2773bf63-kube-api-access-6mj9g\") pod \"maas-api-revoke-sub-0abc5437-8kdbd\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.533943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.533895    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:21.911338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:21.911312    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"]
Oct 30 21:29:21.913042 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:21.913019    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a24106_afed_4372_ab3b_d9ea2773bf63.slice/crio-94c5f25725b923c6d56ab73794679f0302e872e7cf511ace9ee18892e85f7db2 WatchSource:0}: Error finding container 94c5f25725b923c6d56ab73794679f0302e872e7cf511ace9ee18892e85f7db2: Status 404 returned error can't find the container with id 94c5f25725b923c6d56ab73794679f0302e872e7cf511ace9ee18892e85f7db2
Oct 30 21:29:22.355146 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:29:22.355114    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:29:22.801350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:22.801318    5786 generic.go:356] "Generic (PLEG): container finished" podID="e7a24106-afed-4372-ab3b-d9ea2773bf63" containerID="2b49be44b9209e87d5d863f4fa706527a14a54f78bf79b2662ceed56c5a25cc8" exitCode=0
Oct 30 21:29:22.801508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:22.801395    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd" event={"ID":"e7a24106-afed-4372-ab3b-d9ea2773bf63","Type":"ContainerDied","Data":"2b49be44b9209e87d5d863f4fa706527a14a54f78bf79b2662ceed56c5a25cc8"}
Oct 30 21:29:22.801508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:22.801441    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd" event={"ID":"e7a24106-afed-4372-ab3b-d9ea2773bf63","Type":"ContainerStarted","Data":"94c5f25725b923c6d56ab73794679f0302e872e7cf511ace9ee18892e85f7db2"}
Oct 30 21:29:23.950269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:23.950246    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:24.139620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.139549    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7a24106-afed-4372-ab3b-d9ea2773bf63-kube-api-access-6mj9g\" (UniqueName: \"kubernetes.io/projected/e7a24106-afed-4372-ab3b-d9ea2773bf63-kube-api-access-6mj9g\") pod \"e7a24106-afed-4372-ab3b-d9ea2773bf63\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") "
Oct 30 21:29:24.139746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.139618    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-openshift-ca\") pod \"e7a24106-afed-4372-ab3b-d9ea2773bf63\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") "
Oct 30 21:29:24.139804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.139783    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-xks-ca\") pod \"e7a24106-afed-4372-ab3b-d9ea2773bf63\" (UID: \"e7a24106-afed-4372-ab3b-d9ea2773bf63\") "
Oct 30 21:29:24.139951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.139928    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-openshift-ca" pod "e7a24106-afed-4372-ab3b-d9ea2773bf63" (UID: "e7a24106-afed-4372-ab3b-d9ea2773bf63"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:24.140124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.140111    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:24.141742 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.141721    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-xks-ca" pod "e7a24106-afed-4372-ab3b-d9ea2773bf63" (UID: "e7a24106-afed-4372-ab3b-d9ea2773bf63"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:24.141841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.141749    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a24106-afed-4372-ab3b-d9ea2773bf63-kube-api-access-6mj9g" pod "e7a24106-afed-4372-ab3b-d9ea2773bf63" (UID: "e7a24106-afed-4372-ab3b-d9ea2773bf63"). InnerVolumeSpecName "kube-api-access-6mj9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:24.241361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.241340    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mj9g\" (UniqueName: \"kubernetes.io/projected/e7a24106-afed-4372-ab3b-d9ea2773bf63-kube-api-access-6mj9g\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:24.241361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.241360    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a24106-afed-4372-ab3b-d9ea2773bf63-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:24.575638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.575606    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"]
Oct 30 21:29:24.577240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.576189    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e7a24106-afed-4372-ab3b-d9ea2773bf63" containerName="revoke-keys"
Oct 30 21:29:24.577240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.576225    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7a24106-afed-4372-ab3b-d9ea2773bf63" containerName="revoke-keys"
Oct 30 21:29:24.577240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.576433    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7a24106-afed-4372-ab3b-d9ea2773bf63" containerName="revoke-keys"
Oct 30 21:29:24.583229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.583186    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.636473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.636449    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"]
Oct 30 21:29:24.745022 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.744991    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.745157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.745042    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvknt\" (UniqueName: \"kubernetes.io/projected/0b2d7a3b-acaa-4721-975e-7c945cf77345-kube-api-access-vvknt\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.745157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.745100    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-xks-ca\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.814088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.814070    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"
Oct 30 21:29:24.814245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.814094    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd" event={"ID":"e7a24106-afed-4372-ab3b-d9ea2773bf63","Type":"ContainerDied","Data":"94c5f25725b923c6d56ab73794679f0302e872e7cf511ace9ee18892e85f7db2"}
Oct 30 21:29:24.814245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.814132    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94c5f25725b923c6d56ab73794679f0302e872e7cf511ace9ee18892e85f7db2"
Oct 30 21:29:24.846034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.845981    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.846034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.846018    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvknt\" (UniqueName: \"kubernetes.io/projected/0b2d7a3b-acaa-4721-975e-7c945cf77345-kube-api-access-vvknt\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.846165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.846037    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-xks-ca\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.846673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.846653    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.848450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.848424    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-xks-ca\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:24.903407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:24.903382    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvknt\" (UniqueName: \"kubernetes.io/projected/0b2d7a3b-acaa-4721-975e-7c945cf77345-kube-api-access-vvknt\") pod \"maas-api-revoke-sub-944bb9ec-r5tbv\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:25.197009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:25.196981    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:25.569671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:25.569644    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"]
Oct 30 21:29:25.570969 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:25.570941    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b2d7a3b_acaa_4721_975e_7c945cf77345.slice/crio-f3bb43459a438aa78d29f9e8819c78d41ace6363d684c016b000eb3b1c61baf0 WatchSource:0}: Error finding container f3bb43459a438aa78d29f9e8819c78d41ace6363d684c016b000eb3b1c61baf0: Status 404 returned error can't find the container with id f3bb43459a438aa78d29f9e8819c78d41ace6363d684c016b000eb3b1c61baf0
Oct 30 21:29:25.821389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:25.821332    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv" event={"ID":"0b2d7a3b-acaa-4721-975e-7c945cf77345","Type":"ContainerStarted","Data":"f3bb43459a438aa78d29f9e8819c78d41ace6363d684c016b000eb3b1c61baf0"}
Oct 30 21:29:26.835632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:26.835591    5786 generic.go:356] "Generic (PLEG): container finished" podID="0b2d7a3b-acaa-4721-975e-7c945cf77345" containerID="5fe3d1a18d828035d3754a0dfd9f31e3827224e56f5f0c6c986db52c1a154a29" exitCode=0
Oct 30 21:29:26.836092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:26.835733    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv" event={"ID":"0b2d7a3b-acaa-4721-975e-7c945cf77345","Type":"ContainerDied","Data":"5fe3d1a18d828035d3754a0dfd9f31e3827224e56f5f0c6c986db52c1a154a29"}
Oct 30 21:29:27.988254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:27.988232    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:28.183077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.183053    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-xks-ca\") pod \"0b2d7a3b-acaa-4721-975e-7c945cf77345\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") "
Oct 30 21:29:28.183245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.183097    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b2d7a3b-acaa-4721-975e-7c945cf77345-kube-api-access-vvknt\" (UniqueName: \"kubernetes.io/projected/0b2d7a3b-acaa-4721-975e-7c945cf77345-kube-api-access-vvknt\") pod \"0b2d7a3b-acaa-4721-975e-7c945cf77345\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") "
Oct 30 21:29:28.183245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.183186    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-openshift-ca\") pod \"0b2d7a3b-acaa-4721-975e-7c945cf77345\" (UID: \"0b2d7a3b-acaa-4721-975e-7c945cf77345\") "
Oct 30 21:29:28.183567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.183525    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-openshift-ca" pod "0b2d7a3b-acaa-4721-975e-7c945cf77345" (UID: "0b2d7a3b-acaa-4721-975e-7c945cf77345"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:28.183720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.183702    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:28.185199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.185179    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-xks-ca" pod "0b2d7a3b-acaa-4721-975e-7c945cf77345" (UID: "0b2d7a3b-acaa-4721-975e-7c945cf77345"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:28.185278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.185198    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b2d7a3b-acaa-4721-975e-7c945cf77345-kube-api-access-vvknt" pod "0b2d7a3b-acaa-4721-975e-7c945cf77345" (UID: "0b2d7a3b-acaa-4721-975e-7c945cf77345"). InnerVolumeSpecName "kube-api-access-vvknt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:28.284322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.284298    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b2d7a3b-acaa-4721-975e-7c945cf77345-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:28.284322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.284320    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvknt\" (UniqueName: \"kubernetes.io/projected/0b2d7a3b-acaa-4721-975e-7c945cf77345-kube-api-access-vvknt\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:28.848952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.848927    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"
Oct 30 21:29:28.849122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.848967    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv" event={"ID":"0b2d7a3b-acaa-4721-975e-7c945cf77345","Type":"ContainerDied","Data":"f3bb43459a438aa78d29f9e8819c78d41ace6363d684c016b000eb3b1c61baf0"}
Oct 30 21:29:28.849122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:28.849026    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3bb43459a438aa78d29f9e8819c78d41ace6363d684c016b000eb3b1c61baf0"
Oct 30 21:29:37.144143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.144109    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"]
Oct 30 21:29:37.144713 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.144698    5786 cpu_manager.go:414] "containerMap: removing container" podUID="0b2d7a3b-acaa-4721-975e-7c945cf77345" containerName="revoke-keys"
Oct 30 21:29:37.144758 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.144714    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b2d7a3b-acaa-4721-975e-7c945cf77345" containerName="revoke-keys"
Oct 30 21:29:37.144808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.144799    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b2d7a3b-acaa-4721-975e-7c945cf77345" containerName="revoke-keys"
Oct 30 21:29:37.148134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.148119    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.177370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.177345    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9kgj\" (UniqueName: \"kubernetes.io/projected/928bd384-fd3f-4e0a-80b9-201ee5a2a781-kube-api-access-l9kgj\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.177479 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.177376    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.177573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.177555    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-xks-ca\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.203908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.203885    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"]
Oct 30 21:29:37.249705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.249685    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:29:37.249818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.249730    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:29:37.278379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.278351    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9kgj\" (UniqueName: \"kubernetes.io/projected/928bd384-fd3f-4e0a-80b9-201ee5a2a781-kube-api-access-l9kgj\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.278478 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.278382    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.278527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.278496    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-xks-ca\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.279054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.279033    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.280819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.280799    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-xks-ca\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.334073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.334050    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9kgj\" (UniqueName: \"kubernetes.io/projected/928bd384-fd3f-4e0a-80b9-201ee5a2a781-kube-api-access-l9kgj\") pod \"maas-api-revoke-sub-823f4d5f-fsf8q\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.460699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.460681    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:37.842248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.841526    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"]
Oct 30 21:29:37.853087 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:37.853040    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928bd384_fd3f_4e0a_80b9_201ee5a2a781.slice/crio-71e731de4924e162c1291d294a1cdd9fd4814cceed5951333b723257f9b565f8 WatchSource:0}: Error finding container 71e731de4924e162c1291d294a1cdd9fd4814cceed5951333b723257f9b565f8: Status 404 returned error can't find the container with id 71e731de4924e162c1291d294a1cdd9fd4814cceed5951333b723257f9b565f8
Oct 30 21:29:37.904546 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:37.904512    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q" event={"ID":"928bd384-fd3f-4e0a-80b9-201ee5a2a781","Type":"ContainerStarted","Data":"71e731de4924e162c1291d294a1cdd9fd4814cceed5951333b723257f9b565f8"}
Oct 30 21:29:38.932909 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:38.932874    5786 generic.go:356] "Generic (PLEG): container finished" podID="928bd384-fd3f-4e0a-80b9-201ee5a2a781" containerID="8fac0ce7cd1a6366420530298703304af58af7b7129b7ccd3e6bf89707e861b0" exitCode=0
Oct 30 21:29:38.933639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:38.933609    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q" event={"ID":"928bd384-fd3f-4e0a-80b9-201ee5a2a781","Type":"ContainerDied","Data":"8fac0ce7cd1a6366420530298703304af58af7b7129b7ccd3e6bf89707e861b0"}
Oct 30 21:29:40.198995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.198963    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:40.269858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.269779    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-openshift-ca\") pod \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") "
Oct 30 21:29:40.269990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.269922    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/928bd384-fd3f-4e0a-80b9-201ee5a2a781-kube-api-access-l9kgj\" (UniqueName: \"kubernetes.io/projected/928bd384-fd3f-4e0a-80b9-201ee5a2a781-kube-api-access-l9kgj\") pod \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") "
Oct 30 21:29:40.270067 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.270024    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-xks-ca\") pod \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\" (UID: \"928bd384-fd3f-4e0a-80b9-201ee5a2a781\") "
Oct 30 21:29:40.270985 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.270944    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-openshift-ca" pod "928bd384-fd3f-4e0a-80b9-201ee5a2a781" (UID: "928bd384-fd3f-4e0a-80b9-201ee5a2a781"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:40.277775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.277740    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-xks-ca" pod "928bd384-fd3f-4e0a-80b9-201ee5a2a781" (UID: "928bd384-fd3f-4e0a-80b9-201ee5a2a781"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:40.278272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.278241    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928bd384-fd3f-4e0a-80b9-201ee5a2a781-kube-api-access-l9kgj" pod "928bd384-fd3f-4e0a-80b9-201ee5a2a781" (UID: "928bd384-fd3f-4e0a-80b9-201ee5a2a781"). InnerVolumeSpecName "kube-api-access-l9kgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:40.372338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.372311    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9kgj\" (UniqueName: \"kubernetes.io/projected/928bd384-fd3f-4e0a-80b9-201ee5a2a781-kube-api-access-l9kgj\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:40.372495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.372480    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:40.372778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.372761    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/928bd384-fd3f-4e0a-80b9-201ee5a2a781-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:40.985921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.985894    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"
Oct 30 21:29:40.985921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.985916    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q" event={"ID":"928bd384-fd3f-4e0a-80b9-201ee5a2a781","Type":"ContainerDied","Data":"71e731de4924e162c1291d294a1cdd9fd4814cceed5951333b723257f9b565f8"}
Oct 30 21:29:40.986124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:40.985964    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71e731de4924e162c1291d294a1cdd9fd4814cceed5951333b723257f9b565f8"
Oct 30 21:29:41.290602 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.290520    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"]
Oct 30 21:29:41.291303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.291283    5786 cpu_manager.go:414] "containerMap: removing container" podUID="928bd384-fd3f-4e0a-80b9-201ee5a2a781" containerName="revoke-keys"
Oct 30 21:29:41.291303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.291304    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="928bd384-fd3f-4e0a-80b9-201ee5a2a781" containerName="revoke-keys"
Oct 30 21:29:41.291468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.291439    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="928bd384-fd3f-4e0a-80b9-201ee5a2a781" containerName="revoke-keys"
Oct 30 21:29:41.303409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.303381    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.354033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.354003    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"]
Oct 30 21:29:41.354846 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.354820    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:29:41.382780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.382752    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:29:41.384355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.384334    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.384450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.384363    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.384450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.384401    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmxgg\" (UniqueName: \"kubernetes.io/projected/a9b6c27c-6a3b-4429-8de8-26d2f4181697-kube-api-access-pmxgg\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.485888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.485864    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.486004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.485896    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.486004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.485963    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxgg\" (UniqueName: \"kubernetes.io/projected/a9b6c27c-6a3b-4429-8de8-26d2f4181697-kube-api-access-pmxgg\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.486592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.486573    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.488259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.488237    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.542427 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.542365    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmxgg\" (UniqueName: \"kubernetes.io/projected/a9b6c27c-6a3b-4429-8de8-26d2f4181697-kube-api-access-pmxgg\") pod \"maas-api-revoke-sub-8e1546fd-h8brs\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.617869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.617839    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:41.995185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:41.995164    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"]
Oct 30 21:29:41.996618 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:41.996597    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b6c27c_6a3b_4429_8de8_26d2f4181697.slice/crio-a7f71f6939e8f10b53b6018519d53829df0efc2a1bc84bf9c3ac9eff9c21b454 WatchSource:0}: Error finding container a7f71f6939e8f10b53b6018519d53829df0efc2a1bc84bf9c3ac9eff9c21b454: Status 404 returned error can't find the container with id a7f71f6939e8f10b53b6018519d53829df0efc2a1bc84bf9c3ac9eff9c21b454
Oct 30 21:29:43.000718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:43.000691    5786 generic.go:356] "Generic (PLEG): container finished" podID="a9b6c27c-6a3b-4429-8de8-26d2f4181697" containerID="7bba7e233d430188d0dc95641d0ca2019da29593717af2c1316982eb1d7737ef" exitCode=0
Oct 30 21:29:43.001126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:43.000795    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs" event={"ID":"a9b6c27c-6a3b-4429-8de8-26d2f4181697","Type":"ContainerDied","Data":"7bba7e233d430188d0dc95641d0ca2019da29593717af2c1316982eb1d7737ef"}
Oct 30 21:29:43.001126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:43.000861    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs" event={"ID":"a9b6c27c-6a3b-4429-8de8-26d2f4181697","Type":"ContainerStarted","Data":"a7f71f6939e8f10b53b6018519d53829df0efc2a1bc84bf9c3ac9eff9c21b454"}
Oct 30 21:29:44.144138 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.144114    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:44.216372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.216352    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9b6c27c-6a3b-4429-8de8-26d2f4181697-kube-api-access-pmxgg\" (UniqueName: \"kubernetes.io/projected/a9b6c27c-6a3b-4429-8de8-26d2f4181697-kube-api-access-pmxgg\") pod \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") "
Oct 30 21:29:44.216491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.216403    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-openshift-ca\") pod \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") "
Oct 30 21:29:44.216491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.216464    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-xks-ca\") pod \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\" (UID: \"a9b6c27c-6a3b-4429-8de8-26d2f4181697\") "
Oct 30 21:29:44.216808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.216783    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-openshift-ca" pod "a9b6c27c-6a3b-4429-8de8-26d2f4181697" (UID: "a9b6c27c-6a3b-4429-8de8-26d2f4181697"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:44.216982 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.216949    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:44.218484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.218457    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b6c27c-6a3b-4429-8de8-26d2f4181697-kube-api-access-pmxgg" pod "a9b6c27c-6a3b-4429-8de8-26d2f4181697" (UID: "a9b6c27c-6a3b-4429-8de8-26d2f4181697"). InnerVolumeSpecName "kube-api-access-pmxgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:44.218561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.218511    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-xks-ca" pod "a9b6c27c-6a3b-4429-8de8-26d2f4181697" (UID: "a9b6c27c-6a3b-4429-8de8-26d2f4181697"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:44.317651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.317578    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmxgg\" (UniqueName: \"kubernetes.io/projected/a9b6c27c-6a3b-4429-8de8-26d2f4181697-kube-api-access-pmxgg\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:44.317651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:44.317605    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9b6c27c-6a3b-4429-8de8-26d2f4181697-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:45.014694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:45.014671    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"
Oct 30 21:29:45.014925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:45.014667    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs" event={"ID":"a9b6c27c-6a3b-4429-8de8-26d2f4181697","Type":"ContainerDied","Data":"a7f71f6939e8f10b53b6018519d53829df0efc2a1bc84bf9c3ac9eff9c21b454"}
Oct 30 21:29:45.014925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:45.014792    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7f71f6939e8f10b53b6018519d53829df0efc2a1bc84bf9c3ac9eff9c21b454"
Oct 30 21:29:51.933319 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:51.933288    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"]
Oct 30 21:29:51.933736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:51.933478    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy" containerID="cri-o://a79a64b5de6c31af2aaf9ff9fabed5bb99ff12d9d7f342cadbb50464cc8d1472" gracePeriod=30
Oct 30 21:29:52.357855 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:29:52.357779    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:29:56.695130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.695098    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"]
Oct 30 21:29:56.695667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.695653    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a9b6c27c-6a3b-4429-8de8-26d2f4181697" containerName="revoke-keys"
Oct 30 21:29:56.695714 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.695669    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9b6c27c-6a3b-4429-8de8-26d2f4181697" containerName="revoke-keys"
Oct 30 21:29:56.695780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.695772    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9b6c27c-6a3b-4429-8de8-26d2f4181697" containerName="revoke-keys"
Oct 30 21:29:56.700254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.700235    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.757095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.757072    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"]
Oct 30 21:29:56.757225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.757164    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sgjf\" (UniqueName: \"kubernetes.io/projected/770d013e-c334-4993-b248-1c24c7bea1a1-kube-api-access-2sgjf\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.757265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.757239    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.757407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.757390    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.801884 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.801849    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:29:56.802029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.802014    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:29:56.858679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.858656    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sgjf\" (UniqueName: \"kubernetes.io/projected/770d013e-c334-4993-b248-1c24c7bea1a1-kube-api-access-2sgjf\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.858786 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.858735    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.858860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.858845    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.859348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.859324    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.861129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.861112    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:56.916212 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:56.916189    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sgjf\" (UniqueName: \"kubernetes.io/projected/770d013e-c334-4993-b248-1c24c7bea1a1-kube-api-access-2sgjf\") pod \"maas-api-revoke-sub-1db764c1-qhsqm\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:57.012495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.012474    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:29:57.093848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.093817    5786 generic.go:356] "Generic (PLEG): container finished" podID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerID="a79a64b5de6c31af2aaf9ff9fabed5bb99ff12d9d7f342cadbb50464cc8d1472" exitCode=0
Oct 30 21:29:57.094161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.094096    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" event={"ID":"16b786b9-4afd-4f00-b1bc-4af69e49643a","Type":"ContainerDied","Data":"a79a64b5de6c31af2aaf9ff9fabed5bb99ff12d9d7f342cadbb50464cc8d1472"}
Oct 30 21:29:57.195419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.195395    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:29:57.263003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.262940    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-token\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-token\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.262993    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16b786b9-4afd-4f00-b1bc-4af69e49643a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b786b9-4afd-4f00-b1bc-4af69e49643a-istiod-ca-cert\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263040    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-data\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263066    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-envoy\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263131    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-kube-api-access-vsk82\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-kube-api-access-vsk82\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263244    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-socket\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263313    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-data" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:57.263385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263331    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-podinfo\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263349    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16b786b9-4afd-4f00-b1bc-4af69e49643a-istiod-ca-cert" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:57.263385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263372    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-certs\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263428    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-credential-socket\") pod \"16b786b9-4afd-4f00-b1bc-4af69e49643a\" (UID: \"16b786b9-4afd-4f00-b1bc-4af69e49643a\") "
Oct 30 21:29:57.263622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263552    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-socket" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:57.263805 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263764    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-certs" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:57.263896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.263867    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-credential-socket" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:57.264279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.264258    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.264341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.264287    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.264341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.264303    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b786b9-4afd-4f00-b1bc-4af69e49643a-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.264341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.264317    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.264341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.264330    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.265578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.265543    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-podinfo" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:29:57.265681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.265572    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-envoy" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:57.265743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.265685    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-kube-api-access-vsk82" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "kube-api-access-vsk82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:57.265743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.265699    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-token" pod "16b786b9-4afd-4f00-b1bc-4af69e49643a" (UID: "16b786b9-4afd-4f00-b1bc-4af69e49643a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:57.365171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.365141    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsk82\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-kube-api-access-vsk82\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.365171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.365168    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.365171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.365178    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.365411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.365187    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b786b9-4afd-4f00-b1bc-4af69e49643a-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:29:57.432489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:57.431019    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"]
Oct 30 21:29:57.433594 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:29:57.433569    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod770d013e_c334_4993_b248_1c24c7bea1a1.slice/crio-9c4d3d5082ba04b67df414e1879ee1eef782c2e56c5a3b8abd5b4b6d745ada74 WatchSource:0}: Error finding container 9c4d3d5082ba04b67df414e1879ee1eef782c2e56c5a3b8abd5b4b6d745ada74: Status 404 returned error can't find the container with id 9c4d3d5082ba04b67df414e1879ee1eef782c2e56c5a3b8abd5b4b6d745ada74
Oct 30 21:29:58.102724 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:58.102644    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm" event={"ID":"770d013e-c334-4993-b248-1c24c7bea1a1","Type":"ContainerStarted","Data":"d14157cb15f622060ddd0be9e9a118fed78982690d54b5a74deec9d3a22cf9ea"}
Oct 30 21:29:58.102724 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:58.102705    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm" event={"ID":"770d013e-c334-4993-b248-1c24c7bea1a1","Type":"ContainerStarted","Data":"9c4d3d5082ba04b67df414e1879ee1eef782c2e56c5a3b8abd5b4b6d745ada74"}
Oct 30 21:29:58.104088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:58.104071    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"
Oct 30 21:29:58.104158 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:58.104070    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd" event={"ID":"16b786b9-4afd-4f00-b1bc-4af69e49643a","Type":"ContainerDied","Data":"78ce5cbd0ec54677d4657e5914202f47ac317c433659eabc38b502b421a4893b"}
Oct 30 21:29:58.104214 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:58.104192    5786 scope.go:122] "RemoveContainer" containerID="a79a64b5de6c31af2aaf9ff9fabed5bb99ff12d9d7f342cadbb50464cc8d1472"
Oct 30 21:29:58.219264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:58.219226    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm" podStartSLOduration=2.219200752 podStartE2EDuration="2.219200752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:29:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:29:58.215531243 +0000 UTC m=+1054.086650359" watchObservedRunningTime="2026-10-30 21:29:58.219200752 +0000 UTC m=+1054.090319871"
Oct 30 21:29:58.326705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:58.326685    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"]
Oct 30 21:29:58.382845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:58.376669    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-clmqd"]
Oct 30 21:29:59.112610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:59.112573    5786 generic.go:356] "Generic (PLEG): container finished" podID="770d013e-c334-4993-b248-1c24c7bea1a1" containerID="d14157cb15f622060ddd0be9e9a118fed78982690d54b5a74deec9d3a22cf9ea" exitCode=0
Oct 30 21:29:59.112969 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:29:59.112652    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm" event={"ID":"770d013e-c334-4993-b248-1c24c7bea1a1","Type":"ContainerDied","Data":"d14157cb15f622060ddd0be9e9a118fed78982690d54b5a74deec9d3a22cf9ea"}
Oct 30 21:30:00.171273 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.171239    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"]
Oct 30 21:30:00.171922 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.171900    5786 cpu_manager.go:414] "containerMap: removing container" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy"
Oct 30 21:30:00.172080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.172067    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy"
Oct 30 21:30:00.172444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.172430    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" containerName="istio-proxy"
Oct 30 21:30:00.182841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.182819    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"]
Oct 30 21:30:00.182969 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.182950    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:00.186782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.186765    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:00.192874 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.192852    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"]
Oct 30 21:30:00.198055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.198036    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:00.222544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.222512    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"]
Oct 30 21:30:00.227748 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.227730    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:00.274565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.274539    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"]
Oct 30 21:30:00.278458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.278434    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:30:00.296835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.296812    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmdq5\" (UniqueName: \"kubernetes.io/projected/54e2cbcd-2642-4f77-a6c4-e167b2798f58-kube-api-access-dmdq5\") pod \"maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl\" (UID: \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:00.296952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.296850    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dbb56b-5298-40f4-adbe-89341865927b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8\" (UID: \"55dbb56b-5298-40f4-adbe-89341865927b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:00.296952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.296930    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c955c7c-6375-41e5-b75d-c0d3025d7bff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr\" (UID: \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:00.297064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.296952    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57h56\" (UniqueName: \"kubernetes.io/projected/9662e718-905f-46b1-9d6a-cead19fa1d3e-kube-api-access-57h56\") pod \"maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz\" (UID: \"9662e718-905f-46b1-9d6a-cead19fa1d3e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:00.297128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.297091    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbxxt\" (UniqueName: \"kubernetes.io/projected/55dbb56b-5298-40f4-adbe-89341865927b-kube-api-access-vbxxt\") pod \"maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8\" (UID: \"55dbb56b-5298-40f4-adbe-89341865927b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:00.297186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.297132    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9662e718-905f-46b1-9d6a-cead19fa1d3e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz\" (UID: \"9662e718-905f-46b1-9d6a-cead19fa1d3e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:00.297186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.297160    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e2cbcd-2642-4f77-a6c4-e167b2798f58-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl\" (UID: \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:00.297353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.297319    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74t55\" (UniqueName: \"kubernetes.io/projected/5c955c7c-6375-41e5-b75d-c0d3025d7bff-kube-api-access-74t55\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr\" (UID: \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:00.320453 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.320433    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"]
Oct 30 21:30:00.324980 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.324959    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"]
Oct 30 21:30:00.329104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.329065    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"]
Oct 30 21:30:00.360646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.360623    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16b786b9-4afd-4f00-b1bc-4af69e49643a" path="/var/lib/kubelet/pods/16b786b9-4afd-4f00-b1bc-4af69e49643a/volumes"
Oct 30 21:30:00.398706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.398652    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/770d013e-c334-4993-b248-1c24c7bea1a1-kube-api-access-2sgjf\" (UniqueName: \"kubernetes.io/projected/770d013e-c334-4993-b248-1c24c7bea1a1-kube-api-access-2sgjf\") pod \"770d013e-c334-4993-b248-1c24c7bea1a1\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") "
Oct 30 21:30:00.398808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.398795    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-xks-ca\") pod \"770d013e-c334-4993-b248-1c24c7bea1a1\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") "
Oct 30 21:30:00.398923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.398909    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-openshift-ca\") pod \"770d013e-c334-4993-b248-1c24c7bea1a1\" (UID: \"770d013e-c334-4993-b248-1c24c7bea1a1\") "
Oct 30 21:30:00.399080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399064    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74t55\" (UniqueName: \"kubernetes.io/projected/5c955c7c-6375-41e5-b75d-c0d3025d7bff-kube-api-access-74t55\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr\" (UID: \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:00.399199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399185    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmdq5\" (UniqueName: \"kubernetes.io/projected/54e2cbcd-2642-4f77-a6c4-e167b2798f58-kube-api-access-dmdq5\") pod \"maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl\" (UID: \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:00.399290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399185    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-openshift-ca" pod "770d013e-c334-4993-b248-1c24c7bea1a1" (UID: "770d013e-c334-4993-b248-1c24c7bea1a1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:00.399290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399277    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dbb56b-5298-40f4-adbe-89341865927b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8\" (UID: \"55dbb56b-5298-40f4-adbe-89341865927b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:00.399372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399328    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c955c7c-6375-41e5-b75d-c0d3025d7bff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr\" (UID: \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:00.399372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399346    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57h56\" (UniqueName: \"kubernetes.io/projected/9662e718-905f-46b1-9d6a-cead19fa1d3e-kube-api-access-57h56\") pod \"maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz\" (UID: \"9662e718-905f-46b1-9d6a-cead19fa1d3e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:00.399643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399619    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbxxt\" (UniqueName: \"kubernetes.io/projected/55dbb56b-5298-40f4-adbe-89341865927b-kube-api-access-vbxxt\") pod \"maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8\" (UID: \"55dbb56b-5298-40f4-adbe-89341865927b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:00.399712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399664    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9662e718-905f-46b1-9d6a-cead19fa1d3e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz\" (UID: \"9662e718-905f-46b1-9d6a-cead19fa1d3e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:00.399712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399692    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e2cbcd-2642-4f77-a6c4-e167b2798f58-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl\" (UID: \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:00.399820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.399784    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:00.400034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.400015    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dbb56b-5298-40f4-adbe-89341865927b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8\" (UID: \"55dbb56b-5298-40f4-adbe-89341865927b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:00.400133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.400116    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c955c7c-6375-41e5-b75d-c0d3025d7bff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr\" (UID: \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:00.400505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.400479    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e2cbcd-2642-4f77-a6c4-e167b2798f58-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl\" (UID: \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:00.400803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.400785    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9662e718-905f-46b1-9d6a-cead19fa1d3e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz\" (UID: \"9662e718-905f-46b1-9d6a-cead19fa1d3e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:00.401119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.401102    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-xks-ca" pod "770d013e-c334-4993-b248-1c24c7bea1a1" (UID: "770d013e-c334-4993-b248-1c24c7bea1a1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:00.401170 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.401124    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/770d013e-c334-4993-b248-1c24c7bea1a1-kube-api-access-2sgjf" pod "770d013e-c334-4993-b248-1c24c7bea1a1" (UID: "770d013e-c334-4993-b248-1c24c7bea1a1"). InnerVolumeSpecName "kube-api-access-2sgjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:00.456100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.456076    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbxxt\" (UniqueName: \"kubernetes.io/projected/55dbb56b-5298-40f4-adbe-89341865927b-kube-api-access-vbxxt\") pod \"maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8\" (UID: \"55dbb56b-5298-40f4-adbe-89341865927b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:00.456198 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.456107    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57h56\" (UniqueName: \"kubernetes.io/projected/9662e718-905f-46b1-9d6a-cead19fa1d3e-kube-api-access-57h56\") pod \"maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz\" (UID: \"9662e718-905f-46b1-9d6a-cead19fa1d3e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:00.456661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.456641    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74t55\" (UniqueName: \"kubernetes.io/projected/5c955c7c-6375-41e5-b75d-c0d3025d7bff-kube-api-access-74t55\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr\" (UID: \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:00.456698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.456660    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmdq5\" (UniqueName: \"kubernetes.io/projected/54e2cbcd-2642-4f77-a6c4-e167b2798f58-kube-api-access-dmdq5\") pod \"maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl\" (UID: \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:00.500792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.500769    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/770d013e-c334-4993-b248-1c24c7bea1a1-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:00.500792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.500793    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sgjf\" (UniqueName: \"kubernetes.io/projected/770d013e-c334-4993-b248-1c24c7bea1a1-kube-api-access-2sgjf\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:00.508648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.508629    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:00.532773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.532749    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:00.542000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.541978    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:00.550238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.550198    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:00.917033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.917009    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"]
Oct 30 21:30:00.918125 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:30:00.918099    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55dbb56b_5298_40f4_adbe_89341865927b.slice/crio-bff6a426cae27efcf45f32d07bee0985d232d6135278e3577a8a55de48265030 WatchSource:0}: Error finding container bff6a426cae27efcf45f32d07bee0985d232d6135278e3577a8a55de48265030: Status 404 returned error can't find the container with id bff6a426cae27efcf45f32d07bee0985d232d6135278e3577a8a55de48265030
Oct 30 21:30:00.992177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:00.990091    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"]
Oct 30 21:30:01.003349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.003327    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"]
Oct 30 21:30:01.004354 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:30:01.004328    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e2cbcd_2642_4f77_a6c4_e167b2798f58.slice/crio-ea36e0c1c54d2dc600b413aedf55e932ffff2cc9f3d70abe6e1f095d3cbc9ad2 WatchSource:0}: Error finding container ea36e0c1c54d2dc600b413aedf55e932ffff2cc9f3d70abe6e1f095d3cbc9ad2: Status 404 returned error can't find the container with id ea36e0c1c54d2dc600b413aedf55e932ffff2cc9f3d70abe6e1f095d3cbc9ad2
Oct 30 21:30:01.007862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.007837    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"]
Oct 30 21:30:01.126566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.126526    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" event={"ID":"5c955c7c-6375-41e5-b75d-c0d3025d7bff","Type":"ContainerStarted","Data":"900c48166b68dbb6a70cda1f40ca0ad64e38ad66c57bf299e4b3fb79da8dee5e"}
Oct 30 21:30:01.127728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.127701    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz" event={"ID":"9662e718-905f-46b1-9d6a-cead19fa1d3e","Type":"ContainerStarted","Data":"1bb63d0ed6cef45418b43ec7b76f97c6af5c982a870dac5d96976a3b11c58707"}
Oct 30 21:30:01.128991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.128972    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"
Oct 30 21:30:01.128991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.128978    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm" event={"ID":"770d013e-c334-4993-b248-1c24c7bea1a1","Type":"ContainerDied","Data":"9c4d3d5082ba04b67df414e1879ee1eef782c2e56c5a3b8abd5b4b6d745ada74"}
Oct 30 21:30:01.129134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.129019    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c4d3d5082ba04b67df414e1879ee1eef782c2e56c5a3b8abd5b4b6d745ada74"
Oct 30 21:30:01.130190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.130166    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" event={"ID":"55dbb56b-5298-40f4-adbe-89341865927b","Type":"ContainerStarted","Data":"bff6a426cae27efcf45f32d07bee0985d232d6135278e3577a8a55de48265030"}
Oct 30 21:30:01.131508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:01.131482    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" event={"ID":"54e2cbcd-2642-4f77-a6c4-e167b2798f58","Type":"ContainerStarted","Data":"ea36e0c1c54d2dc600b413aedf55e932ffff2cc9f3d70abe6e1f095d3cbc9ad2"}
Oct 30 21:30:02.139582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.139500    5786 generic.go:356] "Generic (PLEG): container finished" podID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerID="f27d85b90b327cb955efa4ef82634e445e5801e624099ed955bc15bfff92f76c" exitCode=60
Oct 30 21:30:02.139975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.139582    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" event={"ID":"54e2cbcd-2642-4f77-a6c4-e167b2798f58","Type":"ContainerDied","Data":"f27d85b90b327cb955efa4ef82634e445e5801e624099ed955bc15bfff92f76c"}
Oct 30 21:30:02.139975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.139757    5786 scope.go:122] "RemoveContainer" containerID="f27d85b90b327cb955efa4ef82634e445e5801e624099ed955bc15bfff92f76c"
Oct 30 21:30:02.141507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.141486    5786 generic.go:356] "Generic (PLEG): container finished" podID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerID="bb1ec8bbd7abbc6ef6a28bb945b42ac66d8ff496d115ab20c21b0e695b29845c" exitCode=60
Oct 30 21:30:02.141625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.141571    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" event={"ID":"5c955c7c-6375-41e5-b75d-c0d3025d7bff","Type":"ContainerDied","Data":"bb1ec8bbd7abbc6ef6a28bb945b42ac66d8ff496d115ab20c21b0e695b29845c"}
Oct 30 21:30:02.141775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.141759    5786 scope.go:122] "RemoveContainer" containerID="bb1ec8bbd7abbc6ef6a28bb945b42ac66d8ff496d115ab20c21b0e695b29845c"
Oct 30 21:30:02.143112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.143092    5786 generic.go:356] "Generic (PLEG): container finished" podID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerID="17637cfeffe194cde08675914d79ea4ea2f14c5175f2cf526bca477b4e2cff74" exitCode=60
Oct 30 21:30:02.143219 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.143175    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz" event={"ID":"9662e718-905f-46b1-9d6a-cead19fa1d3e","Type":"ContainerDied","Data":"17637cfeffe194cde08675914d79ea4ea2f14c5175f2cf526bca477b4e2cff74"}
Oct 30 21:30:02.143372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.143355    5786 scope.go:122] "RemoveContainer" containerID="17637cfeffe194cde08675914d79ea4ea2f14c5175f2cf526bca477b4e2cff74"
Oct 30 21:30:02.144779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.144759    5786 generic.go:356] "Generic (PLEG): container finished" podID="55dbb56b-5298-40f4-adbe-89341865927b" containerID="47a6c44b817c8c7d95d2c042c960dc2756028c5c9c2ce303069dfc5ca6b3507e" exitCode=60
Oct 30 21:30:02.144868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.144821    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" event={"ID":"55dbb56b-5298-40f4-adbe-89341865927b","Type":"ContainerDied","Data":"47a6c44b817c8c7d95d2c042c960dc2756028c5c9c2ce303069dfc5ca6b3507e"}
Oct 30 21:30:02.144961 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:02.144949    5786 scope.go:122] "RemoveContainer" containerID="47a6c44b817c8c7d95d2c042c960dc2756028c5c9c2ce303069dfc5ca6b3507e"
Oct 30 21:30:03.153725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.153682    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" event={"ID":"5c955c7c-6375-41e5-b75d-c0d3025d7bff","Type":"ContainerStarted","Data":"4b4025c7e2210f68e095a868133a394e4ea22468d088751ee1889b8f97424402"}
Oct 30 21:30:03.156593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.156566    5786 generic.go:356] "Generic (PLEG): container finished" podID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerID="cb94389cf22930745e2df23db8ed0d06051985946b043b2f49134f843c5dd195" exitCode=60
Oct 30 21:30:03.156728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.156653    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz" event={"ID":"9662e718-905f-46b1-9d6a-cead19fa1d3e","Type":"ContainerDied","Data":"cb94389cf22930745e2df23db8ed0d06051985946b043b2f49134f843c5dd195"}
Oct 30 21:30:03.156728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.156703    5786 scope.go:122] "RemoveContainer" containerID="17637cfeffe194cde08675914d79ea4ea2f14c5175f2cf526bca477b4e2cff74"
Oct 30 21:30:03.156908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.156883    5786 scope.go:122] "RemoveContainer" containerID="cb94389cf22930745e2df23db8ed0d06051985946b043b2f49134f843c5dd195"
Oct 30 21:30:03.157168 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:03.157146    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz_odh-ai-gateway-infra(9662e718-905f-46b1-9d6a-cead19fa1d3e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e"
Oct 30 21:30:03.159072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.159048    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" event={"ID":"55dbb56b-5298-40f4-adbe-89341865927b","Type":"ContainerStarted","Data":"bdd079d9b82b2151f5110f1bf1404f0fd7e444c52bbfcbb6b071b603dd866e06"}
Oct 30 21:30:03.161021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.161000    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" event={"ID":"54e2cbcd-2642-4f77-a6c4-e167b2798f58","Type":"ContainerStarted","Data":"4b1e7628b95c0139e224f009e0a0b314412b8466d496eaa9ebca3021781418ad"}
Oct 30 21:30:03.269061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.269018    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" podStartSLOduration=3.269003467 podStartE2EDuration="3.269003467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:03.267114746 +0000 UTC m=+1059.138233864" watchObservedRunningTime="2026-10-30 21:30:03.269003467 +0000 UTC m=+1059.140122628"
Oct 30 21:30:03.380750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.380706    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" podStartSLOduration=3.380691467 podStartE2EDuration="3.380691467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:03.379468763 +0000 UTC m=+1059.250587884" watchObservedRunningTime="2026-10-30 21:30:03.380691467 +0000 UTC m=+1059.251810586"
Oct 30 21:30:03.606253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:03.606156    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" podStartSLOduration=3.606143177 podStartE2EDuration="3.606143177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:03.603738222 +0000 UTC m=+1059.474857355" watchObservedRunningTime="2026-10-30 21:30:03.606143177 +0000 UTC m=+1059.477262295"
Oct 30 21:30:04.169993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.169968    5786 generic.go:356] "Generic (PLEG): container finished" podID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerID="4b4025c7e2210f68e095a868133a394e4ea22468d088751ee1889b8f97424402" exitCode=60
Oct 30 21:30:04.170473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.170031    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" event={"ID":"5c955c7c-6375-41e5-b75d-c0d3025d7bff","Type":"ContainerDied","Data":"4b4025c7e2210f68e095a868133a394e4ea22468d088751ee1889b8f97424402"}
Oct 30 21:30:04.170473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.170077    5786 scope.go:122] "RemoveContainer" containerID="bb1ec8bbd7abbc6ef6a28bb945b42ac66d8ff496d115ab20c21b0e695b29845c"
Oct 30 21:30:04.170473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.170317    5786 scope.go:122] "RemoveContainer" containerID="4b4025c7e2210f68e095a868133a394e4ea22468d088751ee1889b8f97424402"
Oct 30 21:30:04.170621 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:04.170550    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr_odh-ai-gateway-infra(5c955c7c-6375-41e5-b75d-c0d3025d7bff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff"
Oct 30 21:30:04.171975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.171957    5786 scope.go:122] "RemoveContainer" containerID="cb94389cf22930745e2df23db8ed0d06051985946b043b2f49134f843c5dd195"
Oct 30 21:30:04.172150 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:04.172134    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz_odh-ai-gateway-infra(9662e718-905f-46b1-9d6a-cead19fa1d3e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e"
Oct 30 21:30:04.173576 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.173550    5786 generic.go:356] "Generic (PLEG): container finished" podID="55dbb56b-5298-40f4-adbe-89341865927b" containerID="bdd079d9b82b2151f5110f1bf1404f0fd7e444c52bbfcbb6b071b603dd866e06" exitCode=60
Oct 30 21:30:04.173678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.173633    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" event={"ID":"55dbb56b-5298-40f4-adbe-89341865927b","Type":"ContainerDied","Data":"bdd079d9b82b2151f5110f1bf1404f0fd7e444c52bbfcbb6b071b603dd866e06"}
Oct 30 21:30:04.173807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.173789    5786 scope.go:122] "RemoveContainer" containerID="bdd079d9b82b2151f5110f1bf1404f0fd7e444c52bbfcbb6b071b603dd866e06"
Oct 30 21:30:04.174005 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:04.173988    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8_odh-ai-gateway-infra(55dbb56b-5298-40f4-adbe-89341865927b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" podUID="55dbb56b-5298-40f4-adbe-89341865927b"
Oct 30 21:30:04.175537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.175519    5786 generic.go:356] "Generic (PLEG): container finished" podID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerID="4b1e7628b95c0139e224f009e0a0b314412b8466d496eaa9ebca3021781418ad" exitCode=60
Oct 30 21:30:04.175629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.175583    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" event={"ID":"54e2cbcd-2642-4f77-a6c4-e167b2798f58","Type":"ContainerDied","Data":"4b1e7628b95c0139e224f009e0a0b314412b8466d496eaa9ebca3021781418ad"}
Oct 30 21:30:04.175734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.175720    5786 scope.go:122] "RemoveContainer" containerID="4b1e7628b95c0139e224f009e0a0b314412b8466d496eaa9ebca3021781418ad"
Oct 30 21:30:04.175893 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:04.175876    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl_odh-ai-gateway-infra(54e2cbcd-2642-4f77-a6c4-e167b2798f58)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58"
Oct 30 21:30:04.182462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.182447    5786 scope.go:122] "RemoveContainer" containerID="47a6c44b817c8c7d95d2c042c960dc2756028c5c9c2ce303069dfc5ca6b3507e"
Oct 30 21:30:04.194682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:04.194662    5786 scope.go:122] "RemoveContainer" containerID="f27d85b90b327cb955efa4ef82634e445e5801e624099ed955bc15bfff92f76c"
Oct 30 21:30:05.183396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:05.183362    5786 scope.go:122] "RemoveContainer" containerID="bdd079d9b82b2151f5110f1bf1404f0fd7e444c52bbfcbb6b071b603dd866e06"
Oct 30 21:30:05.183826 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:05.183626    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8_odh-ai-gateway-infra(55dbb56b-5298-40f4-adbe-89341865927b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" podUID="55dbb56b-5298-40f4-adbe-89341865927b"
Oct 30 21:30:05.184932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:05.184911    5786 scope.go:122] "RemoveContainer" containerID="4b1e7628b95c0139e224f009e0a0b314412b8466d496eaa9ebca3021781418ad"
Oct 30 21:30:05.185110 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:05.185093    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl_odh-ai-gateway-infra(54e2cbcd-2642-4f77-a6c4-e167b2798f58)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58"
Oct 30 21:30:05.186566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:05.186550    5786 scope.go:122] "RemoveContainer" containerID="4b4025c7e2210f68e095a868133a394e4ea22468d088751ee1889b8f97424402"
Oct 30 21:30:05.186734 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:05.186717    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr_odh-ai-gateway-infra(5c955c7c-6375-41e5-b75d-c0d3025d7bff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff"
Oct 30 21:30:13.351990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:13.351964    5786 scope.go:122] "RemoveContainer" containerID="cb94389cf22930745e2df23db8ed0d06051985946b043b2f49134f843c5dd195"
Oct 30 21:30:14.247403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:14.247378    5786 generic.go:356] "Generic (PLEG): container finished" podID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerID="7ad074dca86a7fa2f0c31222fabdfa6c0093a9ec21f6af82e2479c4a3f4d378a" exitCode=60
Oct 30 21:30:14.247545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:14.247419    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz" event={"ID":"9662e718-905f-46b1-9d6a-cead19fa1d3e","Type":"ContainerDied","Data":"7ad074dca86a7fa2f0c31222fabdfa6c0093a9ec21f6af82e2479c4a3f4d378a"}
Oct 30 21:30:14.247545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:14.247476    5786 scope.go:122] "RemoveContainer" containerID="cb94389cf22930745e2df23db8ed0d06051985946b043b2f49134f843c5dd195"
Oct 30 21:30:14.247768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:14.247749    5786 scope.go:122] "RemoveContainer" containerID="7ad074dca86a7fa2f0c31222fabdfa6c0093a9ec21f6af82e2479c4a3f4d378a"
Oct 30 21:30:14.248037 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:14.248013    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz_odh-ai-gateway-infra(9662e718-905f-46b1-9d6a-cead19fa1d3e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e"
Oct 30 21:30:14.352237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:14.352195    5786 scope.go:122] "RemoveContainer" containerID="bdd079d9b82b2151f5110f1bf1404f0fd7e444c52bbfcbb6b071b603dd866e06"
Oct 30 21:30:14.352614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:14.352306    5786 scope.go:122] "RemoveContainer" containerID="4b1e7628b95c0139e224f009e0a0b314412b8466d496eaa9ebca3021781418ad"
Oct 30 21:30:15.257590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.257549    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" event={"ID":"55dbb56b-5298-40f4-adbe-89341865927b","Type":"ContainerStarted","Data":"9d95c51fbd1b6c29d2807bf105aecc80af5c3523d9af7695a8cb55ef22d434d7"}
Oct 30 21:30:15.259712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.259689    5786 generic.go:356] "Generic (PLEG): container finished" podID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerID="6a30153ca4f0771c3781d05801de509ccbbf3e6b490e7bc0c8f2c6e3e9491f1f" exitCode=60
Oct 30 21:30:15.259840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.259756    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" event={"ID":"54e2cbcd-2642-4f77-a6c4-e167b2798f58","Type":"ContainerDied","Data":"6a30153ca4f0771c3781d05801de509ccbbf3e6b490e7bc0c8f2c6e3e9491f1f"}
Oct 30 21:30:15.259840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.259809    5786 scope.go:122] "RemoveContainer" containerID="4b1e7628b95c0139e224f009e0a0b314412b8466d496eaa9ebca3021781418ad"
Oct 30 21:30:15.260119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.260100    5786 scope.go:122] "RemoveContainer" containerID="6a30153ca4f0771c3781d05801de509ccbbf3e6b490e7bc0c8f2c6e3e9491f1f"
Oct 30 21:30:15.260451 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:15.260424    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl_odh-ai-gateway-infra(54e2cbcd-2642-4f77-a6c4-e167b2798f58)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58"
Oct 30 21:30:15.375237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.375198    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"]
Oct 30 21:30:15.524393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.524372    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:15.686138 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.686102    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9662e718-905f-46b1-9d6a-cead19fa1d3e-kube-api-access-57h56\" (UniqueName: \"kubernetes.io/projected/9662e718-905f-46b1-9d6a-cead19fa1d3e-kube-api-access-57h56\") pod \"9662e718-905f-46b1-9d6a-cead19fa1d3e\" (UID: \"9662e718-905f-46b1-9d6a-cead19fa1d3e\") "
Oct 30 21:30:15.686394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.686313    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9662e718-905f-46b1-9d6a-cead19fa1d3e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9662e718-905f-46b1-9d6a-cead19fa1d3e-maas-api-service-ca\") pod \"9662e718-905f-46b1-9d6a-cead19fa1d3e\" (UID: \"9662e718-905f-46b1-9d6a-cead19fa1d3e\") "
Oct 30 21:30:15.686833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.686810    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9662e718-905f-46b1-9d6a-cead19fa1d3e-maas-api-service-ca" pod "9662e718-905f-46b1-9d6a-cead19fa1d3e" (UID: "9662e718-905f-46b1-9d6a-cead19fa1d3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:15.688920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.688863    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9662e718-905f-46b1-9d6a-cead19fa1d3e-kube-api-access-57h56" pod "9662e718-905f-46b1-9d6a-cead19fa1d3e" (UID: "9662e718-905f-46b1-9d6a-cead19fa1d3e"). InnerVolumeSpecName "kube-api-access-57h56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:15.787717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.787685    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9662e718-905f-46b1-9d6a-cead19fa1d3e-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:15.787717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:15.787712    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57h56\" (UniqueName: \"kubernetes.io/projected/9662e718-905f-46b1-9d6a-cead19fa1d3e-kube-api-access-57h56\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:16.269256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.269229    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"
Oct 30 21:30:16.269536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.269264    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz" event={"ID":"9662e718-905f-46b1-9d6a-cead19fa1d3e","Type":"ContainerDied","Data":"1bb63d0ed6cef45418b43ec7b76f97c6af5c982a870dac5d96976a3b11c58707"}
Oct 30 21:30:16.269536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.269323    5786 scope.go:122] "RemoveContainer" containerID="7ad074dca86a7fa2f0c31222fabdfa6c0093a9ec21f6af82e2479c4a3f4d378a"
Oct 30 21:30:16.271349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.271324    5786 generic.go:356] "Generic (PLEG): container finished" podID="55dbb56b-5298-40f4-adbe-89341865927b" containerID="9d95c51fbd1b6c29d2807bf105aecc80af5c3523d9af7695a8cb55ef22d434d7" exitCode=60
Oct 30 21:30:16.271471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.271395    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" event={"ID":"55dbb56b-5298-40f4-adbe-89341865927b","Type":"ContainerDied","Data":"9d95c51fbd1b6c29d2807bf105aecc80af5c3523d9af7695a8cb55ef22d434d7"}
Oct 30 21:30:16.271720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.271695    5786 scope.go:122] "RemoveContainer" containerID="9d95c51fbd1b6c29d2807bf105aecc80af5c3523d9af7695a8cb55ef22d434d7"
Oct 30 21:30:16.271936 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:16.271917    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8_odh-ai-gateway-infra(55dbb56b-5298-40f4-adbe-89341865927b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" podUID="55dbb56b-5298-40f4-adbe-89341865927b"
Oct 30 21:30:16.280052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.280038    5786 scope.go:122] "RemoveContainer" containerID="bdd079d9b82b2151f5110f1bf1404f0fd7e444c52bbfcbb6b071b603dd866e06"
Oct 30 21:30:16.385418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.385392    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"]
Oct 30 21:30:16.500660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.500632    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"]
Oct 30 21:30:16.548090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.547930    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"]
Oct 30 21:30:16.565423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.564390    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-8ce31a-29889930-dtxqz"]
Oct 30 21:30:16.649774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.649749    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:16.798986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.798911    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54e2cbcd-2642-4f77-a6c4-e167b2798f58-kube-api-access-dmdq5\" (UniqueName: \"kubernetes.io/projected/54e2cbcd-2642-4f77-a6c4-e167b2798f58-kube-api-access-dmdq5\") pod \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\" (UID: \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\") "
Oct 30 21:30:16.799127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.799072    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54e2cbcd-2642-4f77-a6c4-e167b2798f58-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e2cbcd-2642-4f77-a6c4-e167b2798f58-maas-api-service-ca\") pod \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\" (UID: \"54e2cbcd-2642-4f77-a6c4-e167b2798f58\") "
Oct 30 21:30:16.799556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.799518    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54e2cbcd-2642-4f77-a6c4-e167b2798f58-maas-api-service-ca" pod "54e2cbcd-2642-4f77-a6c4-e167b2798f58" (UID: "54e2cbcd-2642-4f77-a6c4-e167b2798f58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:16.801172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.801149    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54e2cbcd-2642-4f77-a6c4-e167b2798f58-kube-api-access-dmdq5" pod "54e2cbcd-2642-4f77-a6c4-e167b2798f58" (UID: "54e2cbcd-2642-4f77-a6c4-e167b2798f58"). InnerVolumeSpecName "kube-api-access-dmdq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:16.900866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.900846    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e2cbcd-2642-4f77-a6c4-e167b2798f58-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:16.900866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:16.900868    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmdq5\" (UniqueName: \"kubernetes.io/projected/54e2cbcd-2642-4f77-a6c4-e167b2798f58-kube-api-access-dmdq5\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:17.281077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.281052    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"
Oct 30 21:30:17.281264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.281087    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl" event={"ID":"54e2cbcd-2642-4f77-a6c4-e167b2798f58","Type":"ContainerDied","Data":"ea36e0c1c54d2dc600b413aedf55e932ffff2cc9f3d70abe6e1f095d3cbc9ad2"}
Oct 30 21:30:17.281264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.281145    5786 scope.go:122] "RemoveContainer" containerID="6a30153ca4f0771c3781d05801de509ccbbf3e6b490e7bc0c8f2c6e3e9491f1f"
Oct 30 21:30:17.352575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.352547    5786 scope.go:122] "RemoveContainer" containerID="4b4025c7e2210f68e095a868133a394e4ea22468d088751ee1889b8f97424402"
Oct 30 21:30:17.437884 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.437864    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:17.443617 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.443595    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"]
Oct 30 21:30:17.448696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.448674    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-847c62-29889930-pg8fl"]
Oct 30 21:30:17.607835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.607813    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55dbb56b-5298-40f4-adbe-89341865927b-kube-api-access-vbxxt\" (UniqueName: \"kubernetes.io/projected/55dbb56b-5298-40f4-adbe-89341865927b-kube-api-access-vbxxt\") pod \"55dbb56b-5298-40f4-adbe-89341865927b\" (UID: \"55dbb56b-5298-40f4-adbe-89341865927b\") "
Oct 30 21:30:17.607973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.607870    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55dbb56b-5298-40f4-adbe-89341865927b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dbb56b-5298-40f4-adbe-89341865927b-maas-api-service-ca\") pod \"55dbb56b-5298-40f4-adbe-89341865927b\" (UID: \"55dbb56b-5298-40f4-adbe-89341865927b\") "
Oct 30 21:30:17.608251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.608229    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55dbb56b-5298-40f4-adbe-89341865927b-maas-api-service-ca" pod "55dbb56b-5298-40f4-adbe-89341865927b" (UID: "55dbb56b-5298-40f4-adbe-89341865927b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:17.609899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.609880    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55dbb56b-5298-40f4-adbe-89341865927b-kube-api-access-vbxxt" pod "55dbb56b-5298-40f4-adbe-89341865927b" (UID: "55dbb56b-5298-40f4-adbe-89341865927b"). InnerVolumeSpecName "kube-api-access-vbxxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:17.709299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.709273    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbxxt\" (UniqueName: \"kubernetes.io/projected/55dbb56b-5298-40f4-adbe-89341865927b-kube-api-access-vbxxt\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:17.709299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:17.709296    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dbb56b-5298-40f4-adbe-89341865927b-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:18.291068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.291034    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"
Oct 30 21:30:18.291378 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.291059    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8" event={"ID":"55dbb56b-5298-40f4-adbe-89341865927b","Type":"ContainerDied","Data":"bff6a426cae27efcf45f32d07bee0985d232d6135278e3577a8a55de48265030"}
Oct 30 21:30:18.291378 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.291158    5786 scope.go:122] "RemoveContainer" containerID="9d95c51fbd1b6c29d2807bf105aecc80af5c3523d9af7695a8cb55ef22d434d7"
Oct 30 21:30:18.294659 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.294637    5786 generic.go:356] "Generic (PLEG): container finished" podID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerID="3fb5492af1da51467bc2510d44f5f9703e25e508b0bd4af71c1a9b0dc2db5098" exitCode=60
Oct 30 21:30:18.294768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.294673    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" event={"ID":"5c955c7c-6375-41e5-b75d-c0d3025d7bff","Type":"ContainerDied","Data":"3fb5492af1da51467bc2510d44f5f9703e25e508b0bd4af71c1a9b0dc2db5098"}
Oct 30 21:30:18.295061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.295046    5786 scope.go:122] "RemoveContainer" containerID="3fb5492af1da51467bc2510d44f5f9703e25e508b0bd4af71c1a9b0dc2db5098"
Oct 30 21:30:18.295370 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:18.295347    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr_odh-ai-gateway-infra(5c955c7c-6375-41e5-b75d-c0d3025d7bff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff"
Oct 30 21:30:18.315868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.315851    5786 scope.go:122] "RemoveContainer" containerID="4b4025c7e2210f68e095a868133a394e4ea22468d088751ee1889b8f97424402"
Oct 30 21:30:18.358723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.358702    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" path="/var/lib/kubelet/pods/54e2cbcd-2642-4f77-a6c4-e167b2798f58/volumes"
Oct 30 21:30:18.359123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.359110    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" path="/var/lib/kubelet/pods/9662e718-905f-46b1-9d6a-cead19fa1d3e/volumes"
Oct 30 21:30:18.458334 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.458308    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"]
Oct 30 21:30:18.464409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:18.464378    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-9ac5c5-29889930-t5gb8"]
Oct 30 21:30:19.589160 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:19.589130    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"]
Oct 30 21:30:19.740679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:19.740656    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:19.932431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:19.932397    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c955c7c-6375-41e5-b75d-c0d3025d7bff-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c955c7c-6375-41e5-b75d-c0d3025d7bff-maas-api-service-ca\") pod \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\" (UID: \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\") "
Oct 30 21:30:19.932606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:19.932592    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c955c7c-6375-41e5-b75d-c0d3025d7bff-kube-api-access-74t55\" (UniqueName: \"kubernetes.io/projected/5c955c7c-6375-41e5-b75d-c0d3025d7bff-kube-api-access-74t55\") pod \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\" (UID: \"5c955c7c-6375-41e5-b75d-c0d3025d7bff\") "
Oct 30 21:30:19.932811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:19.932787    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c955c7c-6375-41e5-b75d-c0d3025d7bff-maas-api-service-ca" pod "5c955c7c-6375-41e5-b75d-c0d3025d7bff" (UID: "5c955c7c-6375-41e5-b75d-c0d3025d7bff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:19.933040 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:19.933025    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c955c7c-6375-41e5-b75d-c0d3025d7bff-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:19.934755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:19.934736    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c955c7c-6375-41e5-b75d-c0d3025d7bff-kube-api-access-74t55" pod "5c955c7c-6375-41e5-b75d-c0d3025d7bff" (UID: "5c955c7c-6375-41e5-b75d-c0d3025d7bff"). InnerVolumeSpecName "kube-api-access-74t55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:20.033966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:20.033939    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74t55\" (UniqueName: \"kubernetes.io/projected/5c955c7c-6375-41e5-b75d-c0d3025d7bff-kube-api-access-74t55\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:20.312775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:20.312701    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"
Oct 30 21:30:20.312775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:20.312733    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr" event={"ID":"5c955c7c-6375-41e5-b75d-c0d3025d7bff","Type":"ContainerDied","Data":"900c48166b68dbb6a70cda1f40ca0ad64e38ad66c57bf299e4b3fb79da8dee5e"}
Oct 30 21:30:20.312983 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:20.312814    5786 scope.go:122] "RemoveContainer" containerID="3fb5492af1da51467bc2510d44f5f9703e25e508b0bd4af71c1a9b0dc2db5098"
Oct 30 21:30:20.359754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:20.359729    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55dbb56b-5298-40f4-adbe-89341865927b" path="/var/lib/kubelet/pods/55dbb56b-5298-40f4-adbe-89341865927b/volumes"
Oct 30 21:30:20.496297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:20.496263    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"]
Oct 30 21:30:20.499256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:20.499232    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889930-snpbr"]
Oct 30 21:30:22.356225 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:22.356169    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:30:22.359666 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:22.359641    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" path="/var/lib/kubelet/pods/5c955c7c-6375-41e5-b75d-c0d3025d7bff/volumes"
Oct 30 21:30:35.374146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374075    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"]
Oct 30 21:30:35.374611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374598    5786 cpu_manager.go:414] "containerMap: removing container" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:35.374668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374613    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:35.374668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374623    5786 cpu_manager.go:414] "containerMap: removing container" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:35.374668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374628    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:35.374668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374637    5786 cpu_manager.go:414] "containerMap: removing container" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:35.374668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374641    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:35.374668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374655    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:35.374668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374659    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:35.374668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374667    5786 cpu_manager.go:414] "containerMap: removing container" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374671    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374681    5786 cpu_manager.go:414] "containerMap: removing container" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374685    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374693    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374696    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374717    5786 cpu_manager.go:414] "containerMap: removing container" podUID="770d013e-c334-4993-b248-1c24c7bea1a1" containerName="revoke-keys"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374721    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="770d013e-c334-4993-b248-1c24c7bea1a1" containerName="revoke-keys"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374731    5786 cpu_manager.go:414] "containerMap: removing container" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374735    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374809    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374815    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374820    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374827    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374832    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374839    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374843    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374851    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="770d013e-c334-4993-b248-1c24c7bea1a1" containerName="revoke-keys"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374857    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:35.374943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.374865    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:35.377880 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.377865    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.398744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.398722    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tls-certs\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.398861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.398757    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mmgl\" (UniqueName: \"kubernetes.io/projected/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-kube-api-access-7mmgl\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.398861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.398790    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-model-cache\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.398992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.398937    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-home\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.398992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.398973    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-dshm\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.399101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.399082    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tmp-dir\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.436454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.436422    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"]
Oct 30 21:30:35.480575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.480554    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-38b832\"/\"distinct-w2-38b832-kserve-self-signed-certs\""
Oct 30 21:30:35.500232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500180    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tls-certs\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.500353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500265    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mmgl\" (UniqueName: \"kubernetes.io/projected/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-kube-api-access-7mmgl\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.500353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500320    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-model-cache\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.500462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500385    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-home\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.500462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500412    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-dshm\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.500568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500508    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tmp-dir\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.500751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500731    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-model-cache\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.500751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500743    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-home\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.500862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.500802    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tmp-dir\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.502621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.502600    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-dshm\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.502820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.502801    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tls-certs\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.561538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.561430    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mmgl\" (UniqueName: \"kubernetes.io/projected/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-kube-api-access-7mmgl\") pod \"distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") " pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:35.689639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:35.689610    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:36.084671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:36.084647    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"]
Oct 30 21:30:36.086115 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:30:36.086077    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89f6b2c2_83f6_4962_b811_4b5e5647d5bc.slice/crio-ee71407a9c0fa95f0ae70c754f97da6164dd18aa422160c8e7ff4437fb98deb1 WatchSource:0}: Error finding container ee71407a9c0fa95f0ae70c754f97da6164dd18aa422160c8e7ff4437fb98deb1: Status 404 returned error can't find the container with id ee71407a9c0fa95f0ae70c754f97da6164dd18aa422160c8e7ff4437fb98deb1
Oct 30 21:30:36.415967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:36.415934    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" event={"ID":"89f6b2c2-83f6-4962-b811-4b5e5647d5bc","Type":"ContainerStarted","Data":"eea172e9f44880ac1a70b06c92c382680f201647f98c8a3cc8d54469ad74836f"}
Oct 30 21:30:36.416466 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:36.415977    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" event={"ID":"89f6b2c2-83f6-4962-b811-4b5e5647d5bc","Type":"ContainerStarted","Data":"ee71407a9c0fa95f0ae70c754f97da6164dd18aa422160c8e7ff4437fb98deb1"}
Oct 30 21:30:36.532149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:36.532100    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podStartSLOduration=1.532086304 podStartE2EDuration="1.532086304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:36.530801582 +0000 UTC m=+1092.401920700" watchObservedRunningTime="2026-10-30 21:30:36.532086304 +0000 UTC m=+1092.403205422"
Oct 30 21:30:36.690579 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:36.690487    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:37.323305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:37.323270    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"]
Oct 30 21:30:37.323503 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:37.323478    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy" containerID="cri-o://84472d51931686c18b7144e6c8bd254e51c214ba57cd1ab3c815b6d59a11210a" gracePeriod=30
Oct 30 21:30:42.454609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.454581    5786 generic.go:356] "Generic (PLEG): container finished" podID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerID="84472d51931686c18b7144e6c8bd254e51c214ba57cd1ab3c815b6d59a11210a" exitCode=0
Oct 30 21:30:42.454965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.454650    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" event={"ID":"598e64a0-0e8a-4fe0-b218-2945257f3f74","Type":"ContainerDied","Data":"84472d51931686c18b7144e6c8bd254e51c214ba57cd1ab3c815b6d59a11210a"}
Oct 30 21:30:42.574022 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.574001    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:30:42.701246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701196    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-podinfo\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701316    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-socket\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701351    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-token\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-token\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701396    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/598e64a0-0e8a-4fe0-b218-2945257f3f74-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/598e64a0-0e8a-4fe0-b218-2945257f3f74-istiod-ca-cert\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701465    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-certs\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701503    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-credential-socket\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701593    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-socket" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:42.701714 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701612    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-kube-api-access-t962m\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-kube-api-access-t962m\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701714 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701661    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-envoy\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701737    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-data\") pod \"598e64a0-0e8a-4fe0-b218-2945257f3f74\" (UID: \"598e64a0-0e8a-4fe0-b218-2945257f3f74\") "
Oct 30 21:30:42.701867 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701816    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-credential-socket" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:42.701915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701852    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598e64a0-0e8a-4fe0-b218-2945257f3f74-istiod-ca-cert" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:42.702005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.701979    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-certs" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:42.702112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.702091    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-data" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:42.702340 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.702317    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:42.702410 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.702342    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:42.702524 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.702471    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/598e64a0-0e8a-4fe0-b218-2945257f3f74-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:42.702557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.702528    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:42.702557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.702541    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:42.703468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.703444    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-podinfo" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:42.703896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.703876    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-token" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:42.704199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.704180    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-kube-api-access-t962m" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "kube-api-access-t962m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:42.704199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.704189    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-envoy" pod "598e64a0-0e8a-4fe0-b218-2945257f3f74" (UID: "598e64a0-0e8a-4fe0-b218-2945257f3f74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:42.803976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.803952    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t962m\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-kube-api-access-t962m\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:42.803976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.803978    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:42.804151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.803993    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:42.804151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:42.804007    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/598e64a0-0e8a-4fe0-b218-2945257f3f74-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:30:43.465190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:43.465163    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"
Oct 30 21:30:43.465679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:43.465195    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq" event={"ID":"598e64a0-0e8a-4fe0-b218-2945257f3f74","Type":"ContainerDied","Data":"830a1f47045237f975c9cb0e968f1a479309f304cdbbf0954696c642aa787f73"}
Oct 30 21:30:43.465679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:43.465264    5786 scope.go:122] "RemoveContainer" containerID="84472d51931686c18b7144e6c8bd254e51c214ba57cd1ab3c815b6d59a11210a"
Oct 30 21:30:43.677960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:43.677935    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"]
Oct 30 21:30:43.683830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:43.683803    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v56rq"]
Oct 30 21:30:44.360650 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:44.360614    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" path="/var/lib/kubelet/pods/598e64a0-0e8a-4fe0-b218-2945257f3f74/volumes"
Oct 30 21:30:45.690549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:45.690518    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:45.703074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:45.703052    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:46.493470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:46.493442    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:30:50.255463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.255434    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"]
Oct 30 21:30:50.256049 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256031    5786 cpu_manager.go:414] "containerMap: removing container" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:50.256091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256053    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:50.256091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256074    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:50.256091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256082    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c955c7c-6375-41e5-b75d-c0d3025d7bff" containerName="cleanup"
Oct 30 21:30:50.256181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256135    5786 cpu_manager.go:414] "containerMap: removing container" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy"
Oct 30 21:30:50.256181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256144    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy"
Oct 30 21:30:50.256181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256161    5786 cpu_manager.go:414] "containerMap: removing container" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:50.256181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256166    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:50.256181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256180    5786 cpu_manager.go:414] "containerMap: removing container" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:50.256354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256186    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:50.256354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256298    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="598e64a0-0e8a-4fe0-b218-2945257f3f74" containerName="istio-proxy"
Oct 30 21:30:50.256354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256310    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="55dbb56b-5298-40f4-adbe-89341865927b" containerName="cleanup"
Oct 30 21:30:50.256354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256316    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="54e2cbcd-2642-4f77-a6c4-e167b2798f58" containerName="cleanup"
Oct 30 21:30:50.256354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.256322    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="9662e718-905f-46b1-9d6a-cead19fa1d3e" containerName="cleanup"
Oct 30 21:30:50.261117 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.261094    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.317663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.317634    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"]
Oct 30 21:30:50.364094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.364066    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-847c62\"/\"kube-root-ca.crt\""
Oct 30 21:30:50.364094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.364069    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-847c62\"/\"openshift-service-ca.crt\""
Oct 30 21:30:50.364094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.364085    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-847c62\"/\"default-dockercfg-rrwk7\""
Oct 30 21:30:50.364321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.364110    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-847c62\"/\"distinct-w4-847c62-kserve-self-signed-certs\""
Oct 30 21:30:50.382691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.382669    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-tmp-dir\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.382807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.382697    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-dshm\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.382807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.382746    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-home\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.382919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.382863    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4066a185-16de-48f3-a35f-70f7220c7937-tls-certs\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.382991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.382972    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg2fn\" (UniqueName: \"kubernetes.io/projected/4066a185-16de-48f3-a35f-70f7220c7937-kube-api-access-mg2fn\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.383038 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.383013    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-model-cache\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.483655 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.483628    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-tmp-dir\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.483834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.483662    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-dshm\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.483834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.483722    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-home\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.483834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.483750    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4066a185-16de-48f3-a35f-70f7220c7937-tls-certs\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.483834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.483815    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg2fn\" (UniqueName: \"kubernetes.io/projected/4066a185-16de-48f3-a35f-70f7220c7937-kube-api-access-mg2fn\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.484049 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.483850    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-model-cache\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.484107 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.484054    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-tmp-dir\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.484164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.484110    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-home\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.484281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.484261    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-model-cache\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.486263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.486239    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-dshm\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.486476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.486458    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4066a185-16de-48f3-a35f-70f7220c7937-tls-certs\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.541593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.541537    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg2fn\" (UniqueName: \"kubernetes.io/projected/4066a185-16de-48f3-a35f-70f7220c7937-kube-api-access-mg2fn\") pod \"distinct-w4-847c62-kserve-995ff599-vcppj\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.573059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.573038    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:50.962293 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:50.962263    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"]
Oct 30 21:30:50.963537 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:30:50.963511    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4066a185_16de_48f3_a35f_70f7220c7937.slice/crio-b04945dfa7b882a00dcedea050e41f336ed7133d5dc6043c7fb8fadcf710a8da WatchSource:0}: Error finding container b04945dfa7b882a00dcedea050e41f336ed7133d5dc6043c7fb8fadcf710a8da: Status 404 returned error can't find the container with id b04945dfa7b882a00dcedea050e41f336ed7133d5dc6043c7fb8fadcf710a8da
Oct 30 21:30:51.514902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:51.514875    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" event={"ID":"4066a185-16de-48f3-a35f-70f7220c7937","Type":"ContainerStarted","Data":"50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b"}
Oct 30 21:30:51.515277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:51.514911    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" event={"ID":"4066a185-16de-48f3-a35f-70f7220c7937","Type":"ContainerStarted","Data":"b04945dfa7b882a00dcedea050e41f336ed7133d5dc6043c7fb8fadcf710a8da"}
Oct 30 21:30:51.574233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:51.574192    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:30:51.629420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:30:51.629376    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podStartSLOduration=1.6293631400000002 podStartE2EDuration="1.62936314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:51.627422775 +0000 UTC m=+1107.498541893" watchObservedRunningTime="2026-10-30 21:30:51.62936314 +0000 UTC m=+1107.500482258"
Oct 30 21:30:52.355226 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:30:52.355173    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:31:00.574093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:00.574061    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:31:00.586464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:00.586438    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:31:01.584336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:01.584304    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:31:14.841526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.841495    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"]
Oct 30 21:31:14.845447 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.845425    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:14.927991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.927963    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"]
Oct 30 21:31:14.933620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.933603    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tmp-dir\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:14.933712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.933627    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qw4h\" (UniqueName: \"kubernetes.io/projected/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-kube-api-access-8qw4h\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:14.933712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.933652    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-model-cache\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:14.933779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.933724    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-dshm\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:14.933779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.933759    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tls-certs\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:14.933858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.933847    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-home\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:14.947925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.947902    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-0a4eab\"/\"openshift-service-ca.crt\""
Oct 30 21:31:14.948029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.947942    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-0a4eab\"/\"kube-root-ca.crt\""
Oct 30 21:31:14.948029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.947966    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-0a4eab\"/\"distinct-w5-0a4eab-kserve-self-signed-certs\""
Oct 30 21:31:14.948131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:14.948078    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-0a4eab\"/\"default-dockercfg-vkfnk\""
Oct 30 21:31:15.035345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035320    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tmp-dir\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.035460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035349    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qw4h\" (UniqueName: \"kubernetes.io/projected/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-kube-api-access-8qw4h\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.035460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035372    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-model-cache\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.035460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035406    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-dshm\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.035460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035428    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tls-certs\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.035644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035480    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-home\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.035792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035774    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tmp-dir\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.035854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035791    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-model-cache\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.035931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.035903    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-home\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.037854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.037824    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-dshm\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.037993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.037978    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tls-certs\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.091177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.091158    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qw4h\" (UniqueName: \"kubernetes.io/projected/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-kube-api-access-8qw4h\") pod \"distinct-w5-0a4eab-kserve-67b579955f-nn2z7\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") " pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.158052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.158001    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:15.551531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.551500    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"]
Oct 30 21:31:15.552566 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:31:15.552533    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2eae98b6_4fb9_4e5f_9fe8_dfe21dd1363c.slice/crio-c7fc8a7bde500d4ec09081c5313708fdac44a9c3619ca506bac280bd6c3b77a7 WatchSource:0}: Error finding container c7fc8a7bde500d4ec09081c5313708fdac44a9c3619ca506bac280bd6c3b77a7: Status 404 returned error can't find the container with id c7fc8a7bde500d4ec09081c5313708fdac44a9c3619ca506bac280bd6c3b77a7
Oct 30 21:31:15.694901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.694866    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" event={"ID":"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c","Type":"ContainerStarted","Data":"7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e"}
Oct 30 21:31:15.695040 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.694914    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" event={"ID":"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c","Type":"ContainerStarted","Data":"c7fc8a7bde500d4ec09081c5313708fdac44a9c3619ca506bac280bd6c3b77a7"}
Oct 30 21:31:15.810088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:15.810003    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podStartSLOduration=1.809989777 podStartE2EDuration="1.809989777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:15.80816891 +0000 UTC m=+1131.679288029" watchObservedRunningTime="2026-10-30 21:31:15.809989777 +0000 UTC m=+1131.681108939"
Oct 30 21:31:16.158266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:16.158183    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:20.383041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.383007    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"]
Oct 30 21:31:20.388833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.388806    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.432833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.432803    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb657\" (UniqueName: \"kubernetes.io/projected/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-kube-api-access-mb657\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.432992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.432854    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tls-certs\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.432992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.432950    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-dshm\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.433111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.433002    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tmp-dir\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.433160 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.433145    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-model-cache\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.433224 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.433175    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-home\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.441958 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.441931    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"]
Oct 30 21:31:20.488147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.488114    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f52c3\"/\"distinct-w3-9f52c3-kserve-self-signed-certs\""
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.552966    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-model-cache\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.553021    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-home\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.553111    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb657\" (UniqueName: \"kubernetes.io/projected/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-kube-api-access-mb657\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.553155    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tls-certs\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.553184    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-dshm\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.553232    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tmp-dir\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.555758    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-model-cache\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.556007    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-home\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.561240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.561155    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tls-certs\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.566893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.566863    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tmp-dir\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.570956 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.570913    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-dshm\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.613215 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.613181    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb657\" (UniqueName: \"kubernetes.io/projected/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-kube-api-access-mb657\") pod \"distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") " pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:20.704798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:20.704779    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:21.121412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:21.121390    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"]
Oct 30 21:31:21.122834 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:31:21.122808    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d55bbdf_7957_4ce8_96c7_9a45ed97dd16.slice/crio-db6db8939e7623b4622ed062541813de9d26ceb49244ff94337e3223beab4063 WatchSource:0}: Error finding container db6db8939e7623b4622ed062541813de9d26ceb49244ff94337e3223beab4063: Status 404 returned error can't find the container with id db6db8939e7623b4622ed062541813de9d26ceb49244ff94337e3223beab4063
Oct 30 21:31:21.734410 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:21.734380    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" event={"ID":"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16","Type":"ContainerStarted","Data":"e3a9b532c60ade0f3c9976722af4b8dec00bb49b7ab98965e4e2782ea0c8834f"}
Oct 30 21:31:21.734810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:21.734421    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" event={"ID":"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16","Type":"ContainerStarted","Data":"db6db8939e7623b4622ed062541813de9d26ceb49244ff94337e3223beab4063"}
Oct 30 21:31:21.851496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:21.851450    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podStartSLOduration=1.851437683 podStartE2EDuration="1.851437683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:21.84815416 +0000 UTC m=+1137.719273295" watchObservedRunningTime="2026-10-30 21:31:21.851437683 +0000 UTC m=+1137.722556800"
Oct 30 21:31:22.353652 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:31:22.353616    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:31:22.705853 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:22.705824    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:25.158832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:25.158803    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:25.170687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:25.170668    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:25.770654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:25.770623    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:31:30.705195 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:30.705166    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:30.716992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:30.716972    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:30.805065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:30.805040    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:31:52.355288 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:31:52.355253    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:31:54.169846 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.169820    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"]
Oct 30 21:31:54.175679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.175660    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.233854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.233828    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"]
Oct 30 21:31:54.291178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291147    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.291344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291182    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.291344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291200    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.291344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291235    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.291450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291346    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.291450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291380    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.291513 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291456    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.291551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291518    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.291551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.291538    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4w6t\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-kube-api-access-s4w6t\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.392743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.392712    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4w6t\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-kube-api-access-s4w6t\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.392982 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.392958    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393200    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393273    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393302    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393339    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393341    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393406    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393491    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393551    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.393791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393636    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.394015 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.393995    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.394128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.394105    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.394201 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.394186    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.395670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.395648    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.395797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.395780    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.449870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.449815    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.449956 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.449881    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4w6t\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-kube-api-access-s4w6t\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.488315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.488296    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:54.927260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.927235    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"]
Oct 30 21:31:54.929456 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:31:54.929432    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c4c0d7_3d6b_4123_8892_7a1a5e9b75e4.slice/crio-02941f20da455a1c7c963a485ca3ad3e473efb2f40aaf40731d41f07ffdbeb26 WatchSource:0}: Error finding container 02941f20da455a1c7c963a485ca3ad3e473efb2f40aaf40731d41f07ffdbeb26: Status 404 returned error can't find the container with id 02941f20da455a1c7c963a485ca3ad3e473efb2f40aaf40731d41f07ffdbeb26
Oct 30 21:31:54.931621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.931592    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:31:54.931736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.931649    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:31:54.931736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:54.931678    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:31:55.025187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:55.025149    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" event={"ID":"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4","Type":"ContainerStarted","Data":"7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da"}
Oct 30 21:31:55.025360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:55.025267    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" event={"ID":"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4","Type":"ContainerStarted","Data":"02941f20da455a1c7c963a485ca3ad3e473efb2f40aaf40731d41f07ffdbeb26"}
Oct 30 21:31:55.146554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:55.146510    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podStartSLOduration=1.146496673 podStartE2EDuration="1.146496673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:55.145735094 +0000 UTC m=+1171.016854213" watchObservedRunningTime="2026-10-30 21:31:55.146496673 +0000 UTC m=+1171.017615861"
Oct 30 21:31:55.489229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:55.489176    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:31:55.491025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:55.490992    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:31:55.491151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:55.491037    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 21:31:56.488971 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:56.488941    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:31:56.489184 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:56.489019    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 21:31:57.489483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:57.489448    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:31:57.489898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:57.489506    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 21:31:58.489244 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:58.489197    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:31:58.489490 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:58.489271    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 21:31:59.489099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:59.489066    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:31:59.489526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:31:59.489114    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 21:32:00.488780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:00.488751    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:32:00.488962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:00.488808    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 21:32:01.489051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:01.489023    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:32:01.489494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:01.489070    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 21:32:02.493358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:02.493329    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:32:02.493826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:02.493810    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:32:02.494621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:02.494603    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:32:22.355995 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:32:22.355959    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:32:25.515089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.515050    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"]
Oct 30 21:32:25.519384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.519366    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.572650 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.572618    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:32:25.616460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.616429    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq4rt\" (UniqueName: \"kubernetes.io/projected/632fb0ea-ea0b-4b62-a841-e55f940c3da0-kube-api-access-xq4rt\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.616664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.616469    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.616664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.616580    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.622124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.622093    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:32:25.625260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.625237    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"]
Oct 30 21:32:25.717720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.717680    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq4rt\" (UniqueName: \"kubernetes.io/projected/632fb0ea-ea0b-4b62-a841-e55f940c3da0-kube-api-access-xq4rt\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.718025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.718001    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.719696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.718277    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.721119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.721084    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.727044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.726995    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.777819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.777743    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq4rt\" (UniqueName: \"kubernetes.io/projected/632fb0ea-ea0b-4b62-a841-e55f940c3da0-kube-api-access-xq4rt\") pod \"maas-api-revoke-sub-c8efcd01-ctw49\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:25.832793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:25.832753    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:26.212423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:26.212394    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"]
Oct 30 21:32:26.213180 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:32:26.213153    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632fb0ea_ea0b_4b62_a841_e55f940c3da0.slice/crio-54decb704faa89e9f3a2df6d1bafe39b10f548751c8fa0c4473b2dd9ea958080 WatchSource:0}: Error finding container 54decb704faa89e9f3a2df6d1bafe39b10f548751c8fa0c4473b2dd9ea958080: Status 404 returned error can't find the container with id 54decb704faa89e9f3a2df6d1bafe39b10f548751c8fa0c4473b2dd9ea958080
Oct 30 21:32:26.263272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:26.263236    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49" event={"ID":"632fb0ea-ea0b-4b62-a841-e55f940c3da0","Type":"ContainerStarted","Data":"54decb704faa89e9f3a2df6d1bafe39b10f548751c8fa0c4473b2dd9ea958080"}
Oct 30 21:32:27.271868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:27.271828    5786 generic.go:356] "Generic (PLEG): container finished" podID="632fb0ea-ea0b-4b62-a841-e55f940c3da0" containerID="3f0d76b1fd0f2c8cc79f1de766ebbde810b778afbbeedf93adfe4b086856eb62" exitCode=0
Oct 30 21:32:27.272291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:27.271917    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49" event={"ID":"632fb0ea-ea0b-4b62-a841-e55f940c3da0","Type":"ContainerDied","Data":"3f0d76b1fd0f2c8cc79f1de766ebbde810b778afbbeedf93adfe4b086856eb62"}
Oct 30 21:32:28.425520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.425487    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:28.557359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.557283    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/632fb0ea-ea0b-4b62-a841-e55f940c3da0-kube-api-access-xq4rt\" (UniqueName: \"kubernetes.io/projected/632fb0ea-ea0b-4b62-a841-e55f940c3da0-kube-api-access-xq4rt\") pod \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") "
Oct 30 21:32:28.557359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.557354    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-openshift-ca\") pod \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") "
Oct 30 21:32:28.557547 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.557532    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-xks-ca\") pod \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\" (UID: \"632fb0ea-ea0b-4b62-a841-e55f940c3da0\") "
Oct 30 21:32:28.557810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.557777    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-openshift-ca" pod "632fb0ea-ea0b-4b62-a841-e55f940c3da0" (UID: "632fb0ea-ea0b-4b62-a841-e55f940c3da0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:28.558236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.558195    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.559872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.559841    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/632fb0ea-ea0b-4b62-a841-e55f940c3da0-kube-api-access-xq4rt" pod "632fb0ea-ea0b-4b62-a841-e55f940c3da0" (UID: "632fb0ea-ea0b-4b62-a841-e55f940c3da0"). InnerVolumeSpecName "kube-api-access-xq4rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:28.560094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.560070    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-xks-ca" pod "632fb0ea-ea0b-4b62-a841-e55f940c3da0" (UID: "632fb0ea-ea0b-4b62-a841-e55f940c3da0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:28.659118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.659074    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/632fb0ea-ea0b-4b62-a841-e55f940c3da0-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.659118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:28.659114    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq4rt\" (UniqueName: \"kubernetes.io/projected/632fb0ea-ea0b-4b62-a841-e55f940c3da0-kube-api-access-xq4rt\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:29.287276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:29.287240    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49" event={"ID":"632fb0ea-ea0b-4b62-a841-e55f940c3da0","Type":"ContainerDied","Data":"54decb704faa89e9f3a2df6d1bafe39b10f548751c8fa0c4473b2dd9ea958080"}
Oct 30 21:32:29.287463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:29.287294    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54decb704faa89e9f3a2df6d1bafe39b10f548751c8fa0c4473b2dd9ea958080"
Oct 30 21:32:29.287463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:29.287255    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"
Oct 30 21:32:36.379546 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.379515    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"]
Oct 30 21:32:36.380099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.380086    5786 cpu_manager.go:414] "containerMap: removing container" podUID="632fb0ea-ea0b-4b62-a841-e55f940c3da0" containerName="revoke-keys"
Oct 30 21:32:36.380153 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.380101    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="632fb0ea-ea0b-4b62-a841-e55f940c3da0" containerName="revoke-keys"
Oct 30 21:32:36.380227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.380200    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="632fb0ea-ea0b-4b62-a841-e55f940c3da0" containerName="revoke-keys"
Oct 30 21:32:36.383709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.383685    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.436960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.436926    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:32:36.443796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.443759    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qcmw\" (UniqueName: \"kubernetes.io/projected/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-kube-api-access-6qcmw\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.443968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.443892    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.444095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.444074    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.486238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.486164    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:32:36.491542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.491505    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"]
Oct 30 21:32:36.545957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.545918    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qcmw\" (UniqueName: \"kubernetes.io/projected/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-kube-api-access-6qcmw\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.546174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.546018    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.546174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.546126    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.546841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.546819    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.548728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.548703    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.604621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.604585    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qcmw\" (UniqueName: \"kubernetes.io/projected/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-kube-api-access-6qcmw\") pod \"maas-api-revoke-sub-b473c34a-5dpw8\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:36.698795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:36.698753    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:37.077992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:37.077962    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"]
Oct 30 21:32:37.079301 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:32:37.079270    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8e7f3e_bc12_4062_82df_cc85bd0f7d44.slice/crio-afb2a8e9e72b5ba869aa5271638594b1edda8a84b285ed02b34ed092366be5e7 WatchSource:0}: Error finding container afb2a8e9e72b5ba869aa5271638594b1edda8a84b285ed02b34ed092366be5e7: Status 404 returned error can't find the container with id afb2a8e9e72b5ba869aa5271638594b1edda8a84b285ed02b34ed092366be5e7
Oct 30 21:32:37.344421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:37.344333    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8" event={"ID":"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44","Type":"ContainerStarted","Data":"afb2a8e9e72b5ba869aa5271638594b1edda8a84b285ed02b34ed092366be5e7"}
Oct 30 21:32:38.353899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:38.353821    5786 generic.go:356] "Generic (PLEG): container finished" podID="8f8e7f3e-bc12-4062-82df-cc85bd0f7d44" containerID="63693c7db2f89e971e5f5be0a5e70dfa610303c11986aabec86e272f51827bbd" exitCode=0
Oct 30 21:32:38.360170 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:38.360138    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8" event={"ID":"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44","Type":"ContainerDied","Data":"63693c7db2f89e971e5f5be0a5e70dfa610303c11986aabec86e272f51827bbd"}
Oct 30 21:32:39.507824 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.507798    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:39.583343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.583305    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-xks-ca\") pod \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") "
Oct 30 21:32:39.583535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.583446    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-openshift-ca\") pod \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") "
Oct 30 21:32:39.583535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.583504    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-kube-api-access-6qcmw\" (UniqueName: \"kubernetes.io/projected/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-kube-api-access-6qcmw\") pod \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\" (UID: \"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44\") "
Oct 30 21:32:39.583849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.583819    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-openshift-ca" pod "8f8e7f3e-bc12-4062-82df-cc85bd0f7d44" (UID: "8f8e7f3e-bc12-4062-82df-cc85bd0f7d44"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:39.585709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.585674    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-xks-ca" pod "8f8e7f3e-bc12-4062-82df-cc85bd0f7d44" (UID: "8f8e7f3e-bc12-4062-82df-cc85bd0f7d44"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:39.585862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.585842    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-kube-api-access-6qcmw" pod "8f8e7f3e-bc12-4062-82df-cc85bd0f7d44" (UID: "8f8e7f3e-bc12-4062-82df-cc85bd0f7d44"). InnerVolumeSpecName "kube-api-access-6qcmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:39.684859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.684820    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:39.684859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.684855    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:39.684859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:39.684865    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qcmw\" (UniqueName: \"kubernetes.io/projected/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44-kube-api-access-6qcmw\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:40.373051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:40.373021    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"
Oct 30 21:32:40.373242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:40.373057    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8" event={"ID":"8f8e7f3e-bc12-4062-82df-cc85bd0f7d44","Type":"ContainerDied","Data":"afb2a8e9e72b5ba869aa5271638594b1edda8a84b285ed02b34ed092366be5e7"}
Oct 30 21:32:40.373242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:40.373103    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afb2a8e9e72b5ba869aa5271638594b1edda8a84b285ed02b34ed092366be5e7"
Oct 30 21:32:43.059042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:43.059007    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"]
Oct 30 21:32:43.104098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:43.104069    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c44f461-glf2w"]
Oct 30 21:32:44.359112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:44.359083    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b48e6537-d5eb-4097-9161-22777d9a919e" path="/var/lib/kubelet/pods/b48e6537-d5eb-4097-9161-22777d9a919e/volumes"
Oct 30 21:32:52.355942 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:32:52.355906    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:32:54.460146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.460105    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"]
Oct 30 21:32:54.460921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.460899    5786 cpu_manager.go:414] "containerMap: removing container" podUID="8f8e7f3e-bc12-4062-82df-cc85bd0f7d44" containerName="revoke-keys"
Oct 30 21:32:54.461021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.460923    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f8e7f3e-bc12-4062-82df-cc85bd0f7d44" containerName="revoke-keys"
Oct 30 21:32:54.461078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.461060    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f8e7f3e-bc12-4062-82df-cc85bd0f7d44" containerName="revoke-keys"
Oct 30 21:32:54.464863 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.464841    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.534245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.534180    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"]
Oct 30 21:32:54.573919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.573879    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.574128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.574086    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-xks-ca\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.574264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.574244    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt4nh\" (UniqueName: \"kubernetes.io/projected/b84821b3-6773-419e-9634-f6f6e1f95f03-kube-api-access-kt4nh\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.574873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.574836    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:32:54.574955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.574892    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:32:54.675663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.675630    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-xks-ca\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.675849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.675709    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt4nh\" (UniqueName: \"kubernetes.io/projected/b84821b3-6773-419e-9634-f6f6e1f95f03-kube-api-access-kt4nh\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.675849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.675794    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.676499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.676445    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.678267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.678246    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-xks-ca\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.731959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.731883    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt4nh\" (UniqueName: \"kubernetes.io/projected/b84821b3-6773-419e-9634-f6f6e1f95f03-kube-api-access-kt4nh\") pod \"maas-api-revoke-sub-699d67f6-snvn5\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:54.780799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:54.780759    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:55.161860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:55.161831    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"]
Oct 30 21:32:55.163659 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:32:55.163626    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84821b3_6773_419e_9634_f6f6e1f95f03.slice/crio-8d2f98df7488cc2a506c6979ea3061206b8817745d57a26d920522409be1e82e WatchSource:0}: Error finding container 8d2f98df7488cc2a506c6979ea3061206b8817745d57a26d920522409be1e82e: Status 404 returned error can't find the container with id 8d2f98df7488cc2a506c6979ea3061206b8817745d57a26d920522409be1e82e
Oct 30 21:32:55.478578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:55.478542    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5" event={"ID":"b84821b3-6773-419e-9634-f6f6e1f95f03","Type":"ContainerStarted","Data":"30cfd947c8357fafd694db842cc30ef1d4dcbb662142b6fb013078bfdec50852"}
Oct 30 21:32:55.479045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:55.478608    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5" event={"ID":"b84821b3-6773-419e-9634-f6f6e1f95f03","Type":"ContainerStarted","Data":"8d2f98df7488cc2a506c6979ea3061206b8817745d57a26d920522409be1e82e"}
Oct 30 21:32:55.596233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:55.596172    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5" podStartSLOduration=1.59615671 podStartE2EDuration="1.59615671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:55.593585655 +0000 UTC m=+1231.464704757" watchObservedRunningTime="2026-10-30 21:32:55.59615671 +0000 UTC m=+1231.467275828"
Oct 30 21:32:56.487370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:56.487333    5786 generic.go:356] "Generic (PLEG): container finished" podID="b84821b3-6773-419e-9634-f6f6e1f95f03" containerID="30cfd947c8357fafd694db842cc30ef1d4dcbb662142b6fb013078bfdec50852" exitCode=0
Oct 30 21:32:56.487879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:56.487423    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5" event={"ID":"b84821b3-6773-419e-9634-f6f6e1f95f03","Type":"ContainerDied","Data":"30cfd947c8357fafd694db842cc30ef1d4dcbb662142b6fb013078bfdec50852"}
Oct 30 21:32:57.668671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.668643    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:57.817996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.817892    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-openshift-ca\") pod \"b84821b3-6773-419e-9634-f6f6e1f95f03\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") "
Oct 30 21:32:57.818263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.818240    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b84821b3-6773-419e-9634-f6f6e1f95f03-kube-api-access-kt4nh\" (UniqueName: \"kubernetes.io/projected/b84821b3-6773-419e-9634-f6f6e1f95f03-kube-api-access-kt4nh\") pod \"b84821b3-6773-419e-9634-f6f6e1f95f03\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") "
Oct 30 21:32:57.818358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.818298    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-xks-ca\") pod \"b84821b3-6773-419e-9634-f6f6e1f95f03\" (UID: \"b84821b3-6773-419e-9634-f6f6e1f95f03\") "
Oct 30 21:32:57.818358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.818334    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-openshift-ca" pod "b84821b3-6773-419e-9634-f6f6e1f95f03" (UID: "b84821b3-6773-419e-9634-f6f6e1f95f03"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:57.818704 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.818682    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:57.820577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.820551    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-xks-ca" pod "b84821b3-6773-419e-9634-f6f6e1f95f03" (UID: "b84821b3-6773-419e-9634-f6f6e1f95f03"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:57.820693 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.820602    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b84821b3-6773-419e-9634-f6f6e1f95f03-kube-api-access-kt4nh" pod "b84821b3-6773-419e-9634-f6f6e1f95f03" (UID: "b84821b3-6773-419e-9634-f6f6e1f95f03"). InnerVolumeSpecName "kube-api-access-kt4nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:57.920289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.920241    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt4nh\" (UniqueName: \"kubernetes.io/projected/b84821b3-6773-419e-9634-f6f6e1f95f03-kube-api-access-kt4nh\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:57.920289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:57.920276    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84821b3-6773-419e-9634-f6f6e1f95f03-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:32:58.510157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:58.510130    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"
Oct 30 21:32:58.510358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:58.510128    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5" event={"ID":"b84821b3-6773-419e-9634-f6f6e1f95f03","Type":"ContainerDied","Data":"8d2f98df7488cc2a506c6979ea3061206b8817745d57a26d920522409be1e82e"}
Oct 30 21:32:58.510358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:32:58.510259    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d2f98df7488cc2a506c6979ea3061206b8817745d57a26d920522409be1e82e"
Oct 30 21:33:10.097390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.096930    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"]
Oct 30 21:33:10.103174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.098250    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b84821b3-6773-419e-9634-f6f6e1f95f03" containerName="revoke-keys"
Oct 30 21:33:10.103174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.098274    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b84821b3-6773-419e-9634-f6f6e1f95f03" containerName="revoke-keys"
Oct 30 21:33:10.103174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.098656    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b84821b3-6773-419e-9634-f6f6e1f95f03" containerName="revoke-keys"
Oct 30 21:33:10.104417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.104399    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.156299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.156270    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"]
Oct 30 21:33:10.171283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.171256    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8844\" (UniqueName: \"kubernetes.io/projected/284e5e57-72e5-48a5-b746-3eb81504ff63-kube-api-access-n8844\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.171412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.171332    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.171480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.171449    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.201591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.201557    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:33:10.201591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.201559    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:33:10.273156 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.273131    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8844\" (UniqueName: \"kubernetes.io/projected/284e5e57-72e5-48a5-b746-3eb81504ff63-kube-api-access-n8844\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.273285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.273159    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.273285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.273182    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.273913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.273889    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.275551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.275535    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.329830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.329810    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8844\" (UniqueName: \"kubernetes.io/projected/284e5e57-72e5-48a5-b746-3eb81504ff63-kube-api-access-n8844\") pod \"maas-api-revoke-sub-bd9d54e5-lj8s2\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.418252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.418229    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:10.794329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.794303    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"]
Oct 30 21:33:10.795539 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:33:10.795514    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod284e5e57_72e5_48a5_b746_3eb81504ff63.slice/crio-e0734c7ecd60d5045cac0482f3fb30afc0d4bb008847cdab63854f476d5373fd WatchSource:0}: Error finding container e0734c7ecd60d5045cac0482f3fb30afc0d4bb008847cdab63854f476d5373fd: Status 404 returned error can't find the container with id e0734c7ecd60d5045cac0482f3fb30afc0d4bb008847cdab63854f476d5373fd
Oct 30 21:33:10.887772 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.887744    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"]
Oct 30 21:33:10.892886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.892870    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:10.946895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.946865    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-847c62\"/\"distinct-2-w4-847c62-kserve-self-signed-certs\""
Oct 30 21:33:10.950867 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.950841    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"]
Oct 30 21:33:10.983977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.983946    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gppg\" (UniqueName: \"kubernetes.io/projected/95156f92-fddd-47a2-910b-838e7197a7d8-kube-api-access-7gppg\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:10.984089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.983997    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-dshm\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:10.984089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.984054    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95156f92-fddd-47a2-910b-838e7197a7d8-tls-certs\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:10.984183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.984168    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-home\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:10.984283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.984268    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-tmp-dir\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:10.984395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:10.984381    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-model-cache\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.087556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.087520    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-model-cache\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.087766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.087667    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gppg\" (UniqueName: \"kubernetes.io/projected/95156f92-fddd-47a2-910b-838e7197a7d8-kube-api-access-7gppg\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.087766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.087705    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-dshm\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.087766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.087731    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95156f92-fddd-47a2-910b-838e7197a7d8-tls-certs\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.087926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.087815    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-home\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.087926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.087886    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-tmp-dir\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.088382 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.088357    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-tmp-dir\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.088876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.088853    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-home\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.089109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.089088    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-model-cache\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.093143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.092847    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95156f92-fddd-47a2-910b-838e7197a7d8-tls-certs\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.093143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.093085    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-dshm\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.147929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.147906    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gppg\" (UniqueName: \"kubernetes.io/projected/95156f92-fddd-47a2-910b-838e7197a7d8-kube-api-access-7gppg\") pod \"distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") " pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.204355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.204328    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:11.600888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.600852    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2" event={"ID":"284e5e57-72e5-48a5-b746-3eb81504ff63","Type":"ContainerStarted","Data":"69ee8ca91e3a52f2a69cf90867cac80a764018fe36c17378482dc3ccdccb48a8"}
Oct 30 21:33:11.601062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.600927    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2" event={"ID":"284e5e57-72e5-48a5-b746-3eb81504ff63","Type":"ContainerStarted","Data":"e0734c7ecd60d5045cac0482f3fb30afc0d4bb008847cdab63854f476d5373fd"}
Oct 30 21:33:11.603411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.603391    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"]
Oct 30 21:33:11.605065 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:33:11.605040    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95156f92_fddd_47a2_910b_838e7197a7d8.slice/crio-10dc57cc0cb022da9a27abbe88e6b952d60f72e73029d9b24426da0f7076cc9c WatchSource:0}: Error finding container 10dc57cc0cb022da9a27abbe88e6b952d60f72e73029d9b24426da0f7076cc9c: Status 404 returned error can't find the container with id 10dc57cc0cb022da9a27abbe88e6b952d60f72e73029d9b24426da0f7076cc9c
Oct 30 21:33:11.716924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:11.716882    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2" podStartSLOduration=1.7168691520000001 podStartE2EDuration="1.716869152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:11.714870072 +0000 UTC m=+1247.585989192" watchObservedRunningTime="2026-10-30 21:33:11.716869152 +0000 UTC m=+1247.587988298"
Oct 30 21:33:12.608606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:12.608569    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" event={"ID":"95156f92-fddd-47a2-910b-838e7197a7d8","Type":"ContainerStarted","Data":"a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77"}
Oct 30 21:33:12.608994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:12.608616    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" event={"ID":"95156f92-fddd-47a2-910b-838e7197a7d8","Type":"ContainerStarted","Data":"10dc57cc0cb022da9a27abbe88e6b952d60f72e73029d9b24426da0f7076cc9c"}
Oct 30 21:33:12.610151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:12.610124    5786 generic.go:356] "Generic (PLEG): container finished" podID="284e5e57-72e5-48a5-b746-3eb81504ff63" containerID="69ee8ca91e3a52f2a69cf90867cac80a764018fe36c17378482dc3ccdccb48a8" exitCode=0
Oct 30 21:33:12.610299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:12.610233    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2" event={"ID":"284e5e57-72e5-48a5-b746-3eb81504ff63","Type":"ContainerDied","Data":"69ee8ca91e3a52f2a69cf90867cac80a764018fe36c17378482dc3ccdccb48a8"}
Oct 30 21:33:12.725529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:12.725477    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podStartSLOduration=2.7254636740000002 podStartE2EDuration="2.725463674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:12.723476608 +0000 UTC m=+1248.594595724" watchObservedRunningTime="2026-10-30 21:33:12.725463674 +0000 UTC m=+1248.596582792"
Oct 30 21:33:13.204471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.204441    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:13.757137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.757116    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:13.818248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.818196    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-openshift-ca\") pod \"284e5e57-72e5-48a5-b746-3eb81504ff63\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") "
Oct 30 21:33:13.818394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.818266    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/284e5e57-72e5-48a5-b746-3eb81504ff63-kube-api-access-n8844\" (UniqueName: \"kubernetes.io/projected/284e5e57-72e5-48a5-b746-3eb81504ff63-kube-api-access-n8844\") pod \"284e5e57-72e5-48a5-b746-3eb81504ff63\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") "
Oct 30 21:33:13.818464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.818408    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-xks-ca\") pod \"284e5e57-72e5-48a5-b746-3eb81504ff63\" (UID: \"284e5e57-72e5-48a5-b746-3eb81504ff63\") "
Oct 30 21:33:13.818596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.818575    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-openshift-ca" pod "284e5e57-72e5-48a5-b746-3eb81504ff63" (UID: "284e5e57-72e5-48a5-b746-3eb81504ff63"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:13.818761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.818746    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:13.820430 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.820406    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/284e5e57-72e5-48a5-b746-3eb81504ff63-kube-api-access-n8844" pod "284e5e57-72e5-48a5-b746-3eb81504ff63" (UID: "284e5e57-72e5-48a5-b746-3eb81504ff63"). InnerVolumeSpecName "kube-api-access-n8844". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:13.820529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.820447    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-xks-ca" pod "284e5e57-72e5-48a5-b746-3eb81504ff63" (UID: "284e5e57-72e5-48a5-b746-3eb81504ff63"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:13.919452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.919430    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/284e5e57-72e5-48a5-b746-3eb81504ff63-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:13.919452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:13.919452    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8844\" (UniqueName: \"kubernetes.io/projected/284e5e57-72e5-48a5-b746-3eb81504ff63-kube-api-access-n8844\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:14.624420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:14.624344    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"
Oct 30 21:33:14.624420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:14.624383    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2" event={"ID":"284e5e57-72e5-48a5-b746-3eb81504ff63","Type":"ContainerDied","Data":"e0734c7ecd60d5045cac0482f3fb30afc0d4bb008847cdab63854f476d5373fd"}
Oct 30 21:33:14.624638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:14.624425    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0734c7ecd60d5045cac0482f3fb30afc0d4bb008847cdab63854f476d5373fd"
Oct 30 21:33:15.049845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:15.049814    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"]
Oct 30 21:33:15.099606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:15.099571    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9e5ac9d-hghsn"]
Oct 30 21:33:16.360023 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:16.359993    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be7f6acb-92fd-4e40-9124-15d6613a3cb5" path="/var/lib/kubelet/pods/be7f6acb-92fd-4e40-9124-15d6613a3cb5/volumes"
Oct 30 21:33:21.204752 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:21.204721    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:21.216733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:21.216705    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:21.683070 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:21.683048    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:33:22.354501 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:33:22.354467    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:33:33.238336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.238244    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"]
Oct 30 21:33:33.239935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.239907    5786 cpu_manager.go:414] "containerMap: removing container" podUID="284e5e57-72e5-48a5-b746-3eb81504ff63" containerName="revoke-keys"
Oct 30 21:33:33.240100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.240084    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="284e5e57-72e5-48a5-b746-3eb81504ff63" containerName="revoke-keys"
Oct 30 21:33:33.240376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.240359    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="284e5e57-72e5-48a5-b746-3eb81504ff63" containerName="revoke-keys"
Oct 30 21:33:33.246295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.246278    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.294950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.294918    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"]
Oct 30 21:33:33.392232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.352145    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:33:33.437232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.427763    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.438519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.396199    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:33:33.439009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.438985    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.439162 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.439145    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twrqn\" (UniqueName: \"kubernetes.io/projected/ad0dc3d6-c86f-4d37-8196-f887411d0683-kube-api-access-twrqn\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.547102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.547014    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.547500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.547475    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.547632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.547617    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twrqn\" (UniqueName: \"kubernetes.io/projected/ad0dc3d6-c86f-4d37-8196-f887411d0683-kube-api-access-twrqn\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.566249 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.562515    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.566249 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.563971    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.622238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.611891    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twrqn\" (UniqueName: \"kubernetes.io/projected/ad0dc3d6-c86f-4d37-8196-f887411d0683-kube-api-access-twrqn\") pod \"maas-api-revoke-sub-e269cbb1-tt8jk\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:33.680230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:33.679527    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:34.087544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:34.087517    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"]
Oct 30 21:33:34.089013 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:33:34.088976    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0dc3d6_c86f_4d37_8196_f887411d0683.slice/crio-6a71c2144226751e2930cd46bafc94bf70add55e7413c22afcbf3239106831a3 WatchSource:0}: Error finding container 6a71c2144226751e2930cd46bafc94bf70add55e7413c22afcbf3239106831a3: Status 404 returned error can't find the container with id 6a71c2144226751e2930cd46bafc94bf70add55e7413c22afcbf3239106831a3
Oct 30 21:33:34.781860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:34.781823    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk" event={"ID":"ad0dc3d6-c86f-4d37-8196-f887411d0683","Type":"ContainerStarted","Data":"8d9334116f9ee9a3cbcd21e846b881f186aff7faa4c89c591a5569992e7b3774"}
Oct 30 21:33:34.782260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:34.781888    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk" event={"ID":"ad0dc3d6-c86f-4d37-8196-f887411d0683","Type":"ContainerStarted","Data":"6a71c2144226751e2930cd46bafc94bf70add55e7413c22afcbf3239106831a3"}
Oct 30 21:33:34.896400 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:34.896354    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk" podStartSLOduration=1.896339593 podStartE2EDuration="1.896339593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:34.895467011 +0000 UTC m=+1270.766586130" watchObservedRunningTime="2026-10-30 21:33:34.896339593 +0000 UTC m=+1270.767458711"
Oct 30 21:33:35.050497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:35.050441    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"]
Oct 30 21:33:35.053504 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:35.053483    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-35418c21-2vx6h"]
Oct 30 21:33:35.789508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:35.789478    5786 generic.go:356] "Generic (PLEG): container finished" podID="ad0dc3d6-c86f-4d37-8196-f887411d0683" containerID="8d9334116f9ee9a3cbcd21e846b881f186aff7faa4c89c591a5569992e7b3774" exitCode=0
Oct 30 21:33:35.789902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:35.789554    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk" event={"ID":"ad0dc3d6-c86f-4d37-8196-f887411d0683","Type":"ContainerDied","Data":"8d9334116f9ee9a3cbcd21e846b881f186aff7faa4c89c591a5569992e7b3774"}
Oct 30 21:33:36.359853 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:36.359821    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7b7589e-ffb2-46d5-92e4-181146eb0526" path="/var/lib/kubelet/pods/c7b7589e-ffb2-46d5-92e4-181146eb0526/volumes"
Oct 30 21:33:36.944395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:36.944372    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:36.987418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:36.987396    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-xks-ca\") pod \"ad0dc3d6-c86f-4d37-8196-f887411d0683\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") "
Oct 30 21:33:36.987520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:36.987438    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad0dc3d6-c86f-4d37-8196-f887411d0683-kube-api-access-twrqn\" (UniqueName: \"kubernetes.io/projected/ad0dc3d6-c86f-4d37-8196-f887411d0683-kube-api-access-twrqn\") pod \"ad0dc3d6-c86f-4d37-8196-f887411d0683\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") "
Oct 30 21:33:36.987589 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:36.987578    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-openshift-ca\") pod \"ad0dc3d6-c86f-4d37-8196-f887411d0683\" (UID: \"ad0dc3d6-c86f-4d37-8196-f887411d0683\") "
Oct 30 21:33:36.987912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:36.987888    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-openshift-ca" pod "ad0dc3d6-c86f-4d37-8196-f887411d0683" (UID: "ad0dc3d6-c86f-4d37-8196-f887411d0683"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:36.989448 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:36.989423    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-xks-ca" pod "ad0dc3d6-c86f-4d37-8196-f887411d0683" (UID: "ad0dc3d6-c86f-4d37-8196-f887411d0683"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:36.989538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:36.989502    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad0dc3d6-c86f-4d37-8196-f887411d0683-kube-api-access-twrqn" pod "ad0dc3d6-c86f-4d37-8196-f887411d0683" (UID: "ad0dc3d6-c86f-4d37-8196-f887411d0683"). InnerVolumeSpecName "kube-api-access-twrqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:37.089456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:37.089393    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:37.089456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:37.089425    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twrqn\" (UniqueName: \"kubernetes.io/projected/ad0dc3d6-c86f-4d37-8196-f887411d0683-kube-api-access-twrqn\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:37.089456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:37.089441    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad0dc3d6-c86f-4d37-8196-f887411d0683-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:37.806143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:37.806115    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"
Oct 30 21:33:37.806143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:37.806128    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk" event={"ID":"ad0dc3d6-c86f-4d37-8196-f887411d0683","Type":"ContainerDied","Data":"6a71c2144226751e2930cd46bafc94bf70add55e7413c22afcbf3239106831a3"}
Oct 30 21:33:37.806408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:37.806175    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a71c2144226751e2930cd46bafc94bf70add55e7413c22afcbf3239106831a3"
Oct 30 21:33:39.986869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:39.986830    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"]
Oct 30 21:33:39.987452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:39.987437    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ad0dc3d6-c86f-4d37-8196-f887411d0683" containerName="revoke-keys"
Oct 30 21:33:39.987452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:39.987453    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad0dc3d6-c86f-4d37-8196-f887411d0683" containerName="revoke-keys"
Oct 30 21:33:39.987551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:39.987542    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad0dc3d6-c86f-4d37-8196-f887411d0683" containerName="revoke-keys"
Oct 30 21:33:39.990979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:39.990964    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.093822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.093794    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"]
Oct 30 21:33:40.127568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.127542    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnbjn\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-kube-api-access-fnbjn\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.127702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.127582    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.127702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.127610    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.127826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.127803    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.127874 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.127857    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.127960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.127946    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.127995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.127981    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.128097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.128083    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec004d3-0158-47fa-87e1-b21eb07a8938-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.128128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.128112    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.229781 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.229753    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec004d3-0158-47fa-87e1-b21eb07a8938-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.229925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.229786    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.229925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.229834    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnbjn\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-kube-api-access-fnbjn\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.229925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.229877    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230085 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230064    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230156    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230191    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230238    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-data\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230261    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230300    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230538    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec004d3-0158-47fa-87e1-b21eb07a8938-istiod-ca-cert\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230541    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-certs\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230599    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-credential-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.230776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.230762    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-socket\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.232218 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.232188    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-envoy\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.232534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.232512    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-podinfo\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.286120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.286062    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnbjn\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-kube-api-access-fnbjn\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.286618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.286593    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-token\") pod \"e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") " pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.303611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.303591    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:40.742614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.742590    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"]
Oct 30 21:33:40.743689 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:33:40.743662    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ec004d3_0158_47fa_87e1_b21eb07a8938.slice/crio-4d2480d1635a1ec46a18d94515e13c69c828e8a75d03752e18da9e96631a4ddf WatchSource:0}: Error finding container 4d2480d1635a1ec46a18d94515e13c69c828e8a75d03752e18da9e96631a4ddf: Status 404 returned error can't find the container with id 4d2480d1635a1ec46a18d94515e13c69c828e8a75d03752e18da9e96631a4ddf
Oct 30 21:33:40.746383 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.746349    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:33:40.746469 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.746430    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:33:40.746469 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.746464    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:33:40.831196 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.831167    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" event={"ID":"6ec004d3-0158-47fa-87e1-b21eb07a8938","Type":"ContainerStarted","Data":"daabe7e7bee711ab9e037238787dd3c111e9d0fb4c12970017598ff3f8b45bf7"}
Oct 30 21:33:40.831349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.831250    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" event={"ID":"6ec004d3-0158-47fa-87e1-b21eb07a8938","Type":"ContainerStarted","Data":"4d2480d1635a1ec46a18d94515e13c69c828e8a75d03752e18da9e96631a4ddf"}
Oct 30 21:33:40.958068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:40.958003    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podStartSLOduration=1.957985761 podStartE2EDuration="1.957985761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:40.95688354 +0000 UTC m=+1276.828002685" watchObservedRunningTime="2026-10-30 21:33:40.957985761 +0000 UTC m=+1276.829104883"
Oct 30 21:33:41.304603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:41.304567    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:41.305844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:41.305822    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:33:41.306009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:41.305864    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 21:33:42.304574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:42.304542    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:33:42.304758 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:42.304601    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 21:33:43.050055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:43.050022    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"]
Oct 30 21:33:43.099259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:43.099230    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-111c5e3c-2lqdg"]
Oct 30 21:33:43.304674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:43.304594    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:33:43.304674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:43.304646    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 21:33:44.304312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:44.304272    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:33:44.304488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:44.304332    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 21:33:44.383841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:44.383811    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f" path="/var/lib/kubelet/pods/ee2c92f3-caad-4eb3-b403-c0d9d0eb9b2f/volumes"
Oct 30 21:33:45.304798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:45.304765    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:33:45.305017 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:45.304819    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 21:33:46.304492 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:46.304460    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:33:46.304909 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:46.304513    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 21:33:47.050074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:47.050042    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"]
Oct 30 21:33:47.101575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:47.101535    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e4a80c0d-pv9dr"]
Oct 30 21:33:47.304616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:47.304539    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:33:47.304616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:47.304602    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 21:33:48.308423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:48.308395    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:48.308868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:48.308850    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:48.309592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:48.309568    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:33:48.360438 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:48.360416    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fe0ac47-cc07-4112-a3c3-c9a2884a6b83" path="/var/lib/kubelet/pods/3fe0ac47-cc07-4112-a3c3-c9a2884a6b83/volumes"
Oct 30 21:33:50.134503 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.134474    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"]
Oct 30 21:33:50.139876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.139860    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.179030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.179006    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jgf8\" (UniqueName: \"kubernetes.io/projected/0c8cc2d7-5a53-453a-97be-e361e7da133d-kube-api-access-6jgf8\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.179141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.179086    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.179197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.179184    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.198910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.198882    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"]
Oct 30 21:33:50.240737 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.240716    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:33:50.240737 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.240727    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:33:50.280809 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.280785    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.280933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.280828    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jgf8\" (UniqueName: \"kubernetes.io/projected/0c8cc2d7-5a53-453a-97be-e361e7da133d-kube-api-access-6jgf8\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.280933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.280918    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.281498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.281476    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.283319 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.283299    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.337721 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.337703    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jgf8\" (UniqueName: \"kubernetes.io/projected/0c8cc2d7-5a53-453a-97be-e361e7da133d-kube-api-access-6jgf8\") pod \"maas-api-revoke-sub-d8ace1c9-lbjk7\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.454495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.454474    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:50.830491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.830468    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"]
Oct 30 21:33:50.831534 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:33:50.831511    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c8cc2d7_5a53_453a_97be_e361e7da133d.slice/crio-bba8feed4924218ab439ef39e0aa2c394947af21e39cf60c070e7f5a401d5be7 WatchSource:0}: Error finding container bba8feed4924218ab439ef39e0aa2c394947af21e39cf60c070e7f5a401d5be7: Status 404 returned error can't find the container with id bba8feed4924218ab439ef39e0aa2c394947af21e39cf60c070e7f5a401d5be7
Oct 30 21:33:50.924971 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:50.924945    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7" event={"ID":"0c8cc2d7-5a53-453a-97be-e361e7da133d","Type":"ContainerStarted","Data":"bba8feed4924218ab439ef39e0aa2c394947af21e39cf60c070e7f5a401d5be7"}
Oct 30 21:33:51.923775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:51.923747    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"]
Oct 30 21:33:51.927965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:51.927947    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:51.933767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:51.933744    5786 generic.go:356] "Generic (PLEG): container finished" podID="0c8cc2d7-5a53-453a-97be-e361e7da133d" containerID="8165bab7250c804f7a045281e71bb243e8d3953d3470fdfdc9cac9b70664d6e9" exitCode=0
Oct 30 21:33:51.933882 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:51.933817    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7" event={"ID":"0c8cc2d7-5a53-453a-97be-e361e7da133d","Type":"ContainerDied","Data":"8165bab7250c804f7a045281e71bb243e8d3953d3470fdfdc9cac9b70664d6e9"}
Oct 30 21:33:51.985889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:51.985839    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"]
Oct 30 21:33:52.009994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.009969    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.010185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.010161    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.010265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.010224    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvvkv\" (UniqueName: \"kubernetes.io/projected/8f5aff79-0f83-4de4-91f1-4410180655c4-kube-api-access-tvvkv\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.111458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.111433    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.111588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.111518    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.111588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.111538    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvvkv\" (UniqueName: \"kubernetes.io/projected/8f5aff79-0f83-4de4-91f1-4410180655c4-kube-api-access-tvvkv\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.112106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.112082    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.113842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.113824    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.167705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.167679    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvvkv\" (UniqueName: \"kubernetes.io/projected/8f5aff79-0f83-4de4-91f1-4410180655c4-kube-api-access-tvvkv\") pod \"maas-api-revoke-sub-cf9e350d-2kssw\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.241738 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.241686    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:52.359153 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:33:52.359041    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:33:52.621546 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.621517    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"]
Oct 30 21:33:52.941800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:52.941771    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw" event={"ID":"8f5aff79-0f83-4de4-91f1-4410180655c4","Type":"ContainerStarted","Data":"a19db3e6b6e4825f7efb594f4fce8e23ca59a67d050af8adf76cf97438b6f2be"}
Oct 30 21:33:53.093378 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.093356    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:53.123403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.123351    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-openshift-ca\") pod \"0c8cc2d7-5a53-453a-97be-e361e7da133d\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") "
Oct 30 21:33:53.123540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.123517    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c8cc2d7-5a53-453a-97be-e361e7da133d-kube-api-access-6jgf8\" (UniqueName: \"kubernetes.io/projected/0c8cc2d7-5a53-453a-97be-e361e7da133d-kube-api-access-6jgf8\") pod \"0c8cc2d7-5a53-453a-97be-e361e7da133d\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") "
Oct 30 21:33:53.123667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.123651    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-xks-ca\") pod \"0c8cc2d7-5a53-453a-97be-e361e7da133d\" (UID: \"0c8cc2d7-5a53-453a-97be-e361e7da133d\") "
Oct 30 21:33:53.123736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.123717    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-openshift-ca" pod "0c8cc2d7-5a53-453a-97be-e361e7da133d" (UID: "0c8cc2d7-5a53-453a-97be-e361e7da133d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:53.124090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.124071    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:53.125540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.125518    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c8cc2d7-5a53-453a-97be-e361e7da133d-kube-api-access-6jgf8" pod "0c8cc2d7-5a53-453a-97be-e361e7da133d" (UID: "0c8cc2d7-5a53-453a-97be-e361e7da133d"). InnerVolumeSpecName "kube-api-access-6jgf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:53.125628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.125563    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-xks-ca" pod "0c8cc2d7-5a53-453a-97be-e361e7da133d" (UID: "0c8cc2d7-5a53-453a-97be-e361e7da133d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:53.225549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.225523    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jgf8\" (UniqueName: \"kubernetes.io/projected/0c8cc2d7-5a53-453a-97be-e361e7da133d-kube-api-access-6jgf8\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:53.225549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.225545    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8cc2d7-5a53-453a-97be-e361e7da133d-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:53.950261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.950237    5786 generic.go:356] "Generic (PLEG): container finished" podID="8f5aff79-0f83-4de4-91f1-4410180655c4" containerID="61d2f01407d55d6b7655cc68163948bc52971430ad89ba71c10bc161cde22840" exitCode=0
Oct 30 21:33:53.950652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.950332    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw" event={"ID":"8f5aff79-0f83-4de4-91f1-4410180655c4","Type":"ContainerDied","Data":"61d2f01407d55d6b7655cc68163948bc52971430ad89ba71c10bc161cde22840"}
Oct 30 21:33:53.951606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.951591    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"
Oct 30 21:33:53.951702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.951635    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7" event={"ID":"0c8cc2d7-5a53-453a-97be-e361e7da133d","Type":"ContainerDied","Data":"bba8feed4924218ab439ef39e0aa2c394947af21e39cf60c070e7f5a401d5be7"}
Oct 30 21:33:53.951702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:53.951674    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bba8feed4924218ab439ef39e0aa2c394947af21e39cf60c070e7f5a401d5be7"
Oct 30 21:33:55.109562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.109535    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:55.151952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.151924    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-xks-ca\") pod \"8f5aff79-0f83-4de4-91f1-4410180655c4\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") "
Oct 30 21:33:55.152114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.152096    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f5aff79-0f83-4de4-91f1-4410180655c4-kube-api-access-tvvkv\" (UniqueName: \"kubernetes.io/projected/8f5aff79-0f83-4de4-91f1-4410180655c4-kube-api-access-tvvkv\") pod \"8f5aff79-0f83-4de4-91f1-4410180655c4\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") "
Oct 30 21:33:55.152301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.152283    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-openshift-ca\") pod \"8f5aff79-0f83-4de4-91f1-4410180655c4\" (UID: \"8f5aff79-0f83-4de4-91f1-4410180655c4\") "
Oct 30 21:33:55.152627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.152565    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-openshift-ca" pod "8f5aff79-0f83-4de4-91f1-4410180655c4" (UID: "8f5aff79-0f83-4de4-91f1-4410180655c4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:55.152627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.152653    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:55.154176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.154156    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-xks-ca" pod "8f5aff79-0f83-4de4-91f1-4410180655c4" (UID: "8f5aff79-0f83-4de4-91f1-4410180655c4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:55.154297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.154222    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f5aff79-0f83-4de4-91f1-4410180655c4-kube-api-access-tvvkv" pod "8f5aff79-0f83-4de4-91f1-4410180655c4" (UID: "8f5aff79-0f83-4de4-91f1-4410180655c4"). InnerVolumeSpecName "kube-api-access-tvvkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:55.254114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.254042    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f5aff79-0f83-4de4-91f1-4410180655c4-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:55.254114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.254077    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvvkv\" (UniqueName: \"kubernetes.io/projected/8f5aff79-0f83-4de4-91f1-4410180655c4-kube-api-access-tvvkv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:33:55.966194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.966161    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"
Oct 30 21:33:55.966369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.966238    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw" event={"ID":"8f5aff79-0f83-4de4-91f1-4410180655c4","Type":"ContainerDied","Data":"a19db3e6b6e4825f7efb594f4fce8e23ca59a67d050af8adf76cf97438b6f2be"}
Oct 30 21:33:55.966369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:55.966279    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a19db3e6b6e4825f7efb594f4fce8e23ca59a67d050af8adf76cf97438b6f2be"
Oct 30 21:33:58.213083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.213055    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"]
Oct 30 21:33:58.213635 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.213621    5786 cpu_manager.go:414] "containerMap: removing container" podUID="8f5aff79-0f83-4de4-91f1-4410180655c4" containerName="revoke-keys"
Oct 30 21:33:58.213680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.213636    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f5aff79-0f83-4de4-91f1-4410180655c4" containerName="revoke-keys"
Oct 30 21:33:58.213680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.213677    5786 cpu_manager.go:414] "containerMap: removing container" podUID="0c8cc2d7-5a53-453a-97be-e361e7da133d" containerName="revoke-keys"
Oct 30 21:33:58.213745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.213683    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c8cc2d7-5a53-453a-97be-e361e7da133d" containerName="revoke-keys"
Oct 30 21:33:58.213790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.213781    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c8cc2d7-5a53-453a-97be-e361e7da133d" containerName="revoke-keys"
Oct 30 21:33:58.213824 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.213795    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f5aff79-0f83-4de4-91f1-4410180655c4" containerName="revoke-keys"
Oct 30 21:33:58.217399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.217379    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.283241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.283186    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.283412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.283326    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.283483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.283466    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58tlk\" (UniqueName: \"kubernetes.io/projected/818ee224-eaab-49a5-8fe6-bcb2196fe626-kube-api-access-58tlk\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.322496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.322472    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:33:58.322496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.322472    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:33:58.325861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.325844    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"]
Oct 30 21:33:58.384871 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.384849    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.384997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.384931    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58tlk\" (UniqueName: \"kubernetes.io/projected/818ee224-eaab-49a5-8fe6-bcb2196fe626-kube-api-access-58tlk\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.385038 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.385014    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.385601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.385571    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.387439 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.387414    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.442324 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.442297    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58tlk\" (UniqueName: \"kubernetes.io/projected/818ee224-eaab-49a5-8fe6-bcb2196fe626-kube-api-access-58tlk\") pod \"maas-api-revoke-sub-c0c6a5c7-w24hw\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.530604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.530552    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:33:58.915629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.915605    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"]
Oct 30 21:33:58.916439 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:33:58.916416    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818ee224_eaab_49a5_8fe6_bcb2196fe626.slice/crio-5e1b7c5a3181993ded2a9108ca63f30d908c64946efceb8325ac93dc5fdf70dc WatchSource:0}: Error finding container 5e1b7c5a3181993ded2a9108ca63f30d908c64946efceb8325ac93dc5fdf70dc: Status 404 returned error can't find the container with id 5e1b7c5a3181993ded2a9108ca63f30d908c64946efceb8325ac93dc5fdf70dc
Oct 30 21:33:58.986126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:58.986101    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw" event={"ID":"818ee224-eaab-49a5-8fe6-bcb2196fe626","Type":"ContainerStarted","Data":"5e1b7c5a3181993ded2a9108ca63f30d908c64946efceb8325ac93dc5fdf70dc"}
Oct 30 21:33:59.063039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:59.063020    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"]
Oct 30 21:33:59.068931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:33:59.068911    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06ae89ef-rtbxq"]
Oct 30 21:34:00.012592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:00.010074    5786 generic.go:356] "Generic (PLEG): container finished" podID="818ee224-eaab-49a5-8fe6-bcb2196fe626" containerID="9f01e366d2e47e44590a12ca4c4e65747c753c20d1fb3db3b15c1e589dd4184e" exitCode=0
Oct 30 21:34:00.012592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:00.010285    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw" event={"ID":"818ee224-eaab-49a5-8fe6-bcb2196fe626","Type":"ContainerDied","Data":"9f01e366d2e47e44590a12ca4c4e65747c753c20d1fb3db3b15c1e589dd4184e"}
Oct 30 21:34:00.359821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:00.359732    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9fcf672-636f-4587-b829-c0989b4018fc" path="/var/lib/kubelet/pods/a9fcf672-636f-4587-b829-c0989b4018fc/volumes"
Oct 30 21:34:01.171651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.171629    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:34:01.221059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.221039    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-xks-ca\") pod \"818ee224-eaab-49a5-8fe6-bcb2196fe626\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") "
Oct 30 21:34:01.221180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.221086    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-openshift-ca\") pod \"818ee224-eaab-49a5-8fe6-bcb2196fe626\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") "
Oct 30 21:34:01.221180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.221113    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/818ee224-eaab-49a5-8fe6-bcb2196fe626-kube-api-access-58tlk\" (UniqueName: \"kubernetes.io/projected/818ee224-eaab-49a5-8fe6-bcb2196fe626-kube-api-access-58tlk\") pod \"818ee224-eaab-49a5-8fe6-bcb2196fe626\" (UID: \"818ee224-eaab-49a5-8fe6-bcb2196fe626\") "
Oct 30 21:34:01.221464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.221439    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-openshift-ca" pod "818ee224-eaab-49a5-8fe6-bcb2196fe626" (UID: "818ee224-eaab-49a5-8fe6-bcb2196fe626"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:01.221739 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.221720    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:01.223291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.223265    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-xks-ca" pod "818ee224-eaab-49a5-8fe6-bcb2196fe626" (UID: "818ee224-eaab-49a5-8fe6-bcb2196fe626"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:01.223370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.223292    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818ee224-eaab-49a5-8fe6-bcb2196fe626-kube-api-access-58tlk" pod "818ee224-eaab-49a5-8fe6-bcb2196fe626" (UID: "818ee224-eaab-49a5-8fe6-bcb2196fe626"). InnerVolumeSpecName "kube-api-access-58tlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:01.322674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.322616    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/818ee224-eaab-49a5-8fe6-bcb2196fe626-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:01.322674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:01.322641    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58tlk\" (UniqueName: \"kubernetes.io/projected/818ee224-eaab-49a5-8fe6-bcb2196fe626-kube-api-access-58tlk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:02.024698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:02.024671    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"
Oct 30 21:34:02.024698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:02.024675    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw" event={"ID":"818ee224-eaab-49a5-8fe6-bcb2196fe626","Type":"ContainerDied","Data":"5e1b7c5a3181993ded2a9108ca63f30d908c64946efceb8325ac93dc5fdf70dc"}
Oct 30 21:34:02.024924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:02.024723    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e1b7c5a3181993ded2a9108ca63f30d908c64946efceb8325ac93dc5fdf70dc"
Oct 30 21:34:06.049009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.048980    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"]
Oct 30 21:34:06.098184 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.098150    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-055ecf74-snllc"]
Oct 30 21:34:06.359348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.359284    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cedb2623-365a-4fa1-b904-657fc204b8ea" path="/var/lib/kubelet/pods/cedb2623-365a-4fa1-b904-657fc204b8ea/volumes"
Oct 30 21:34:06.858339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.858310    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"]
Oct 30 21:34:06.858865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.858853    5786 cpu_manager.go:414] "containerMap: removing container" podUID="818ee224-eaab-49a5-8fe6-bcb2196fe626" containerName="revoke-keys"
Oct 30 21:34:06.858906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.858867    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="818ee224-eaab-49a5-8fe6-bcb2196fe626" containerName="revoke-keys"
Oct 30 21:34:06.858988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.858979    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="818ee224-eaab-49a5-8fe6-bcb2196fe626" containerName="revoke-keys"
Oct 30 21:34:06.862483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.862467    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:06.920152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.920128    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"]
Oct 30 21:34:06.965289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.965254    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:34:06.965289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:06.965254    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:34:07.023228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.023182    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.023346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.023308    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf7qk\" (UniqueName: \"kubernetes.io/projected/b16ec9e7-0734-438f-95df-5734fa44d405-kube-api-access-nf7qk\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.023384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.023349    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.124201 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.124142    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.124570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.124200    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf7qk\" (UniqueName: \"kubernetes.io/projected/b16ec9e7-0734-438f-95df-5734fa44d405-kube-api-access-nf7qk\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.124570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.124253    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.124735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.124716    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.126541 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.126525    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.180089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.180067    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf7qk\" (UniqueName: \"kubernetes.io/projected/b16ec9e7-0734-438f-95df-5734fa44d405-kube-api-access-nf7qk\") pod \"maas-api-revoke-sub-b7bd362e-4dp5g\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.475326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.475303    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:07.854848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:07.854823    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"]
Oct 30 21:34:07.856013 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:07.855945    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16ec9e7_0734_438f_95df_5734fa44d405.slice/crio-83c8ab6c813a9f3425f4783394003b377e13b72e0ee03e580cea8c84cad25eb9 WatchSource:0}: Error finding container 83c8ab6c813a9f3425f4783394003b377e13b72e0ee03e580cea8c84cad25eb9: Status 404 returned error can't find the container with id 83c8ab6c813a9f3425f4783394003b377e13b72e0ee03e580cea8c84cad25eb9
Oct 30 21:34:08.091577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:08.091548    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g" event={"ID":"b16ec9e7-0734-438f-95df-5734fa44d405","Type":"ContainerStarted","Data":"83c8ab6c813a9f3425f4783394003b377e13b72e0ee03e580cea8c84cad25eb9"}
Oct 30 21:34:09.100005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.099974    5786 generic.go:356] "Generic (PLEG): container finished" podID="b16ec9e7-0734-438f-95df-5734fa44d405" containerID="9a011a5c9bbbabc1be4328bbf03d1ad3f999a52c96158dac71bc82100d03f01b" exitCode=0
Oct 30 21:34:09.100411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.100064    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g" event={"ID":"b16ec9e7-0734-438f-95df-5734fa44d405","Type":"ContainerDied","Data":"9a011a5c9bbbabc1be4328bbf03d1ad3f999a52c96158dac71bc82100d03f01b"}
Oct 30 21:34:09.267925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.267892    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"]
Oct 30 21:34:09.272194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.272176    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.344357 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.344326    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"]
Oct 30 21:34:09.347862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.347833    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.348036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.347889    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwmls\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-kube-api-access-gwmls\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.348036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.347941    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.348036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.347980    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.348036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.348014    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.348338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.348036    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.348338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.348111    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9712d6e-b3fd-4be6-8711-e401a193579d-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.348338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.348201    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.348338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.348249    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.449626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449603    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.449787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449636    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwmls\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-kube-api-access-gwmls\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.449787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449666    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.449787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449705    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.449787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449725    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.449787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449743    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.450095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449809    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9712d6e-b3fd-4be6-8711-e401a193579d-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.450095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449889    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.450095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.449916    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.450095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.450017    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.450304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.450097    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.450408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.450304    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.450408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.450333    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.450712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.450690    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9712d6e-b3fd-4be6-8711-e401a193579d-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.452186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.452165    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.452751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.452733    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.507914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.507883    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwmls\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-kube-api-access-gwmls\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.508011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.507970    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:09.585712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:09.585683    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:10.013726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.013696    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"]
Oct 30 21:34:10.014353 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:10.014324    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9712d6e_b3fd_4be6_8711_e401a193579d.slice/crio-5affa8ca1ee90b17bb5dd3943cb0684b165854445615e64d709b7ed2e6f1f64e WatchSource:0}: Error finding container 5affa8ca1ee90b17bb5dd3943cb0684b165854445615e64d709b7ed2e6f1f64e: Status 404 returned error can't find the container with id 5affa8ca1ee90b17bb5dd3943cb0684b165854445615e64d709b7ed2e6f1f64e
Oct 30 21:34:10.016592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.016560    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:10.016688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.016618    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:10.016688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.016653    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:10.111027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.110991    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" event={"ID":"e9712d6e-b3fd-4be6-8711-e401a193579d","Type":"ContainerStarted","Data":"5a046ac40b95bf39d444cec9b753e4a4a1bdbf3b9b85bd1c61c50126f598bf1b"}
Oct 30 21:34:10.111464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.111054    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" event={"ID":"e9712d6e-b3fd-4be6-8711-e401a193579d","Type":"ContainerStarted","Data":"5affa8ca1ee90b17bb5dd3943cb0684b165854445615e64d709b7ed2e6f1f64e"}
Oct 30 21:34:10.228233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.228049    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podStartSLOduration=1.2280297550000001 podStartE2EDuration="1.228029755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:10.22675344 +0000 UTC m=+1306.097872570" watchObservedRunningTime="2026-10-30 21:34:10.228029755 +0000 UTC m=+1306.099148874"
Oct 30 21:34:10.264645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.264578    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:10.371924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.371897    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-openshift-ca\") pod \"b16ec9e7-0734-438f-95df-5734fa44d405\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") "
Oct 30 21:34:10.372110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.372025    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b16ec9e7-0734-438f-95df-5734fa44d405-kube-api-access-nf7qk\" (UniqueName: \"kubernetes.io/projected/b16ec9e7-0734-438f-95df-5734fa44d405-kube-api-access-nf7qk\") pod \"b16ec9e7-0734-438f-95df-5734fa44d405\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") "
Oct 30 21:34:10.372181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.372113    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-xks-ca\") pod \"b16ec9e7-0734-438f-95df-5734fa44d405\" (UID: \"b16ec9e7-0734-438f-95df-5734fa44d405\") "
Oct 30 21:34:10.372343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.372318    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-openshift-ca" pod "b16ec9e7-0734-438f-95df-5734fa44d405" (UID: "b16ec9e7-0734-438f-95df-5734fa44d405"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:10.372605 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.372585    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:10.374431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.374406    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-xks-ca" pod "b16ec9e7-0734-438f-95df-5734fa44d405" (UID: "b16ec9e7-0734-438f-95df-5734fa44d405"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:10.374719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.374696    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b16ec9e7-0734-438f-95df-5734fa44d405-kube-api-access-nf7qk" pod "b16ec9e7-0734-438f-95df-5734fa44d405" (UID: "b16ec9e7-0734-438f-95df-5734fa44d405"). InnerVolumeSpecName "kube-api-access-nf7qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:10.474127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.474105    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf7qk\" (UniqueName: \"kubernetes.io/projected/b16ec9e7-0734-438f-95df-5734fa44d405-kube-api-access-nf7qk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:10.474285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.474193    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b16ec9e7-0734-438f-95df-5734fa44d405-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:10.585901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:10.585826    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:11.119574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:11.119549    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"
Oct 30 21:34:11.119574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:11.119554    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g" event={"ID":"b16ec9e7-0734-438f-95df-5734fa44d405","Type":"ContainerDied","Data":"83c8ab6c813a9f3425f4783394003b377e13b72e0ee03e580cea8c84cad25eb9"}
Oct 30 21:34:11.120045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:11.119606    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83c8ab6c813a9f3425f4783394003b377e13b72e0ee03e580cea8c84cad25eb9"
Oct 30 21:34:11.587757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:11.587724    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:34:11.587965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:11.587771    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 21:34:12.586327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:12.586293    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:34:12.586802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:12.586359    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 21:34:13.586814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:13.586782    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:34:13.587248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:13.586836    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 21:34:14.080842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.080782    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"]
Oct 30 21:34:14.118160 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.118120    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31ba00c4-wtd54"]
Oct 30 21:34:14.381892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.381817    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6d7de61-b323-40c5-8b7b-c0726b871be0" path="/var/lib/kubelet/pods/b6d7de61-b323-40c5-8b7b-c0726b871be0/volumes"
Oct 30 21:34:14.587593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.587557    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:34:14.587978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.587630    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 21:34:14.812522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.812496    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"]
Oct 30 21:34:14.813289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.813271    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b16ec9e7-0734-438f-95df-5734fa44d405" containerName="revoke-keys"
Oct 30 21:34:14.813419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.813406    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b16ec9e7-0734-438f-95df-5734fa44d405" containerName="revoke-keys"
Oct 30 21:34:14.813950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.813729    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b16ec9e7-0734-438f-95df-5734fa44d405" containerName="revoke-keys"
Oct 30 21:34:14.819221 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.819190    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.824149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.824129    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.824305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.824292    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j5f6\" (UniqueName: \"kubernetes.io/projected/19baf13a-39c2-430b-9ae4-c46a2d483a4c-kube-api-access-9j5f6\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.824544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.824533    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.880142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.880114    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"]
Oct 30 21:34:14.918901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.918876    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:34:14.919034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.918946    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:34:14.925429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.925407    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.925552 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.925469    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.925552 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.925491    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j5f6\" (UniqueName: \"kubernetes.io/projected/19baf13a-39c2-430b-9ae4-c46a2d483a4c-kube-api-access-9j5f6\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.926293 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.926272    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.928374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.928352    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:14.982453 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:14.982425    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j5f6\" (UniqueName: \"kubernetes.io/projected/19baf13a-39c2-430b-9ae4-c46a2d483a4c-kube-api-access-9j5f6\") pod \"maas-api-revoke-sub-9322e825-dqrfm\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:15.132935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:15.132866    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:15.542068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:15.542031    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"]
Oct 30 21:34:15.543046 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:15.543020    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19baf13a_39c2_430b_9ae4_c46a2d483a4c.slice/crio-03b7592200575d8ce92bef46bb9102d0eb172d466eebad035f05f34214ba446f WatchSource:0}: Error finding container 03b7592200575d8ce92bef46bb9102d0eb172d466eebad035f05f34214ba446f: Status 404 returned error can't find the container with id 03b7592200575d8ce92bef46bb9102d0eb172d466eebad035f05f34214ba446f
Oct 30 21:34:15.586534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:15.586508    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:34:15.586683 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:15.586550    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 21:34:16.157197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:16.157122    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm" event={"ID":"19baf13a-39c2-430b-9ae4-c46a2d483a4c","Type":"ContainerStarted","Data":"1f7b512a124f07e4ca8026c04c63130ca94393d2d96e4095960945bf39d24888"}
Oct 30 21:34:16.157197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:16.157182    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm" event={"ID":"19baf13a-39c2-430b-9ae4-c46a2d483a4c","Type":"ContainerStarted","Data":"03b7592200575d8ce92bef46bb9102d0eb172d466eebad035f05f34214ba446f"}
Oct 30 21:34:16.271122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:16.271066    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm" podStartSLOduration=2.27104915 podStartE2EDuration="2.27104915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:16.269852932 +0000 UTC m=+1312.140972050" watchObservedRunningTime="2026-10-30 21:34:16.27104915 +0000 UTC m=+1312.142168269"
Oct 30 21:34:16.586470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:16.586443    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:34:16.586620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:16.586494    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 21:34:17.051264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:17.051234    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"]
Oct 30 21:34:17.100114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:17.100079    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b67865fd-75sgr"]
Oct 30 21:34:17.165521 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:17.165494    5786 generic.go:356] "Generic (PLEG): container finished" podID="19baf13a-39c2-430b-9ae4-c46a2d483a4c" containerID="1f7b512a124f07e4ca8026c04c63130ca94393d2d96e4095960945bf39d24888" exitCode=0
Oct 30 21:34:17.165955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:17.165577    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm" event={"ID":"19baf13a-39c2-430b-9ae4-c46a2d483a4c","Type":"ContainerDied","Data":"1f7b512a124f07e4ca8026c04c63130ca94393d2d96e4095960945bf39d24888"}
Oct 30 21:34:17.590432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:17.590406    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:17.590872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:17.590853    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:17.591496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:17.591476    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:34:18.054857 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.054824    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"]
Oct 30 21:34:18.057791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.057771    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a00c2a6f-4nzw5"]
Oct 30 21:34:18.307909 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.307855    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:18.359106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.359080    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ccf67ccb-ccb7-4691-a39f-6d2191eeae74" path="/var/lib/kubelet/pods/ccf67ccb-ccb7-4691-a39f-6d2191eeae74/volumes"
Oct 30 21:34:18.359578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.359562    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="deba8191-7edf-47a7-8218-390d486e5e15" path="/var/lib/kubelet/pods/deba8191-7edf-47a7-8218-390d486e5e15/volumes"
Oct 30 21:34:18.370548 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.370526    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19baf13a-39c2-430b-9ae4-c46a2d483a4c-kube-api-access-9j5f6\" (UniqueName: \"kubernetes.io/projected/19baf13a-39c2-430b-9ae4-c46a2d483a4c-kube-api-access-9j5f6\") pod \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") "
Oct 30 21:34:18.370618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.370597    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-xks-ca\") pod \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") "
Oct 30 21:34:18.370684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.370673    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-openshift-ca\") pod \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\" (UID: \"19baf13a-39c2-430b-9ae4-c46a2d483a4c\") "
Oct 30 21:34:18.370970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.370946    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-openshift-ca" pod "19baf13a-39c2-430b-9ae4-c46a2d483a4c" (UID: "19baf13a-39c2-430b-9ae4-c46a2d483a4c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:18.372707 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.372686    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19baf13a-39c2-430b-9ae4-c46a2d483a4c-kube-api-access-9j5f6" pod "19baf13a-39c2-430b-9ae4-c46a2d483a4c" (UID: "19baf13a-39c2-430b-9ae4-c46a2d483a4c"). InnerVolumeSpecName "kube-api-access-9j5f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:18.372801 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.372707    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-xks-ca" pod "19baf13a-39c2-430b-9ae4-c46a2d483a4c" (UID: "19baf13a-39c2-430b-9ae4-c46a2d483a4c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:18.472744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.472715    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j5f6\" (UniqueName: \"kubernetes.io/projected/19baf13a-39c2-430b-9ae4-c46a2d483a4c-kube-api-access-9j5f6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:18.472744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.472740    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:18.472744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:18.472749    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19baf13a-39c2-430b-9ae4-c46a2d483a4c-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:19.180456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:19.180424    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"
Oct 30 21:34:19.180456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:19.180445    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm" event={"ID":"19baf13a-39c2-430b-9ae4-c46a2d483a4c","Type":"ContainerDied","Data":"03b7592200575d8ce92bef46bb9102d0eb172d466eebad035f05f34214ba446f"}
Oct 30 21:34:19.180698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:19.180492    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03b7592200575d8ce92bef46bb9102d0eb172d466eebad035f05f34214ba446f"
Oct 30 21:34:21.067738 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.067704    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"]
Oct 30 21:34:21.069676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.069646    5786 cpu_manager.go:414] "containerMap: removing container" podUID="19baf13a-39c2-430b-9ae4-c46a2d483a4c" containerName="revoke-keys"
Oct 30 21:34:21.069852 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.069838    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19baf13a-39c2-430b-9ae4-c46a2d483a4c" containerName="revoke-keys"
Oct 30 21:34:21.070265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.070248    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="19baf13a-39c2-430b-9ae4-c46a2d483a4c" containerName="revoke-keys"
Oct 30 21:34:21.075754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.075734    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.075754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.075747    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"]
Oct 30 21:34:21.079601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.079582    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.127475 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.127453    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:34:21.127592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.127577    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:34:21.169503 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.169473    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"]
Oct 30 21:34:21.180082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.180061    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"]
Oct 30 21:34:21.225865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.225833    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.225966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.225890    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.226041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.226029    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpvvc\" (UniqueName: \"kubernetes.io/projected/b8520185-8f72-472c-bc54-a8dfb09da2db-kube-api-access-jpvvc\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.226089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.226066    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.226132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.226091    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.226239 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.226222    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2jbv\" (UniqueName: \"kubernetes.io/projected/5347e77d-10d8-49f4-b60f-9e8250af963d-kube-api-access-c2jbv\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.327099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.327043    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpvvc\" (UniqueName: \"kubernetes.io/projected/b8520185-8f72-472c-bc54-a8dfb09da2db-kube-api-access-jpvvc\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.327099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.327084    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.327320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.327103    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.327320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.327175    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2jbv\" (UniqueName: \"kubernetes.io/projected/5347e77d-10d8-49f4-b60f-9e8250af963d-kube-api-access-c2jbv\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.327320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.327233    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.327320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.327286    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.327865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.327837    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.327961 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.327866    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.329555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.329536    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.329681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.329665    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.384081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.384064    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpvvc\" (UniqueName: \"kubernetes.io/projected/b8520185-8f72-472c-bc54-a8dfb09da2db-kube-api-access-jpvvc\") pod \"maas-api-revoke-sub-b8732eae-7qlnc\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.384175 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.384064    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2jbv\" (UniqueName: \"kubernetes.io/projected/5347e77d-10d8-49f4-b60f-9e8250af963d-kube-api-access-c2jbv\") pod \"maas-api-revoke-sub-c026a502-h4sds\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.390922 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.390898    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:21.397055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.397038    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:21.770872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.770844    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"]
Oct 30 21:34:21.771670 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:21.771640    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5347e77d_10d8_49f4_b60f_9e8250af963d.slice/crio-4bb9268646bf5800143e8803006fc732b5b9ca90dae0e2881990bec1ec9cf5cc WatchSource:0}: Error finding container 4bb9268646bf5800143e8803006fc732b5b9ca90dae0e2881990bec1ec9cf5cc: Status 404 returned error can't find the container with id 4bb9268646bf5800143e8803006fc732b5b9ca90dae0e2881990bec1ec9cf5cc
Oct 30 21:34:21.821557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:21.821536    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"]
Oct 30 21:34:21.822191 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:21.822164    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8520185_8f72_472c_bc54_a8dfb09da2db.slice/crio-c832c612f0e22d1ba8e8b436a9bf2ccd343c09e183520245e42a94ed7a1902a5 WatchSource:0}: Error finding container c832c612f0e22d1ba8e8b436a9bf2ccd343c09e183520245e42a94ed7a1902a5: Status 404 returned error can't find the container with id c832c612f0e22d1ba8e8b436a9bf2ccd343c09e183520245e42a94ed7a1902a5
Oct 30 21:34:22.203494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:22.203468    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc" event={"ID":"b8520185-8f72-472c-bc54-a8dfb09da2db","Type":"ContainerStarted","Data":"fa0471cc13ce873ae9d4c85f16bd99e748f503f0128c9d3169e397517af59abe"}
Oct 30 21:34:22.203845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:22.203522    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc" event={"ID":"b8520185-8f72-472c-bc54-a8dfb09da2db","Type":"ContainerStarted","Data":"c832c612f0e22d1ba8e8b436a9bf2ccd343c09e183520245e42a94ed7a1902a5"}
Oct 30 21:34:22.205172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:22.205147    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds" event={"ID":"5347e77d-10d8-49f4-b60f-9e8250af963d","Type":"ContainerStarted","Data":"5c3221203f1c5f02c4d1b876ad4575975f104ff6a9b9f894c6764a151c8bea96"}
Oct 30 21:34:22.205294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:22.205200    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds" event={"ID":"5347e77d-10d8-49f4-b60f-9e8250af963d","Type":"ContainerStarted","Data":"4bb9268646bf5800143e8803006fc732b5b9ca90dae0e2881990bec1ec9cf5cc"}
Oct 30 21:34:22.319865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:22.319792    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc" podStartSLOduration=1.3197800210000001 podStartE2EDuration="1.319780021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:22.317548386 +0000 UTC m=+1318.188667505" watchObservedRunningTime="2026-10-30 21:34:22.319780021 +0000 UTC m=+1318.190899159"
Oct 30 21:34:22.356148 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:34:22.356121    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:34:22.435853 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:22.435812    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds" podStartSLOduration=1.435799293 podStartE2EDuration="1.435799293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:22.433351408 +0000 UTC m=+1318.304470529" watchObservedRunningTime="2026-10-30 21:34:22.435799293 +0000 UTC m=+1318.306918412"
Oct 30 21:34:23.213654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:23.213622    5786 generic.go:356] "Generic (PLEG): container finished" podID="5347e77d-10d8-49f4-b60f-9e8250af963d" containerID="5c3221203f1c5f02c4d1b876ad4575975f104ff6a9b9f894c6764a151c8bea96" exitCode=0
Oct 30 21:34:23.214023 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:23.213676    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds" event={"ID":"5347e77d-10d8-49f4-b60f-9e8250af963d","Type":"ContainerDied","Data":"5c3221203f1c5f02c4d1b876ad4575975f104ff6a9b9f894c6764a151c8bea96"}
Oct 30 21:34:23.215526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:23.215505    5786 generic.go:356] "Generic (PLEG): container finished" podID="b8520185-8f72-472c-bc54-a8dfb09da2db" containerID="fa0471cc13ce873ae9d4c85f16bd99e748f503f0128c9d3169e397517af59abe" exitCode=0
Oct 30 21:34:23.215623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:23.215538    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc" event={"ID":"b8520185-8f72-472c-bc54-a8dfb09da2db","Type":"ContainerDied","Data":"fa0471cc13ce873ae9d4c85f16bd99e748f503f0128c9d3169e397517af59abe"}
Oct 30 21:34:24.348088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.348058    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"]
Oct 30 21:34:24.423952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.417256    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.447927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.445502    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"]
Oct 30 21:34:24.485464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.485042    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:24.488760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.488739    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:24.566381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566358    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-xks-ca\") pod \"b8520185-8f72-472c-bc54-a8dfb09da2db\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") "
Oct 30 21:34:24.566527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566408    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-openshift-ca\") pod \"5347e77d-10d8-49f4-b60f-9e8250af963d\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") "
Oct 30 21:34:24.566527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566429    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8520185-8f72-472c-bc54-a8dfb09da2db-kube-api-access-jpvvc\" (UniqueName: \"kubernetes.io/projected/b8520185-8f72-472c-bc54-a8dfb09da2db-kube-api-access-jpvvc\") pod \"b8520185-8f72-472c-bc54-a8dfb09da2db\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") "
Oct 30 21:34:24.566527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566514    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5347e77d-10d8-49f4-b60f-9e8250af963d-kube-api-access-c2jbv\" (UniqueName: \"kubernetes.io/projected/5347e77d-10d8-49f4-b60f-9e8250af963d-kube-api-access-c2jbv\") pod \"5347e77d-10d8-49f4-b60f-9e8250af963d\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") "
Oct 30 21:34:24.566681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566553    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-xks-ca\") pod \"5347e77d-10d8-49f4-b60f-9e8250af963d\" (UID: \"5347e77d-10d8-49f4-b60f-9e8250af963d\") "
Oct 30 21:34:24.566681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566621    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-openshift-ca\") pod \"b8520185-8f72-472c-bc54-a8dfb09da2db\" (UID: \"b8520185-8f72-472c-bc54-a8dfb09da2db\") "
Oct 30 21:34:24.566787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566728    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-openshift-ca" pod "5347e77d-10d8-49f4-b60f-9e8250af963d" (UID: "5347e77d-10d8-49f4-b60f-9e8250af963d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:24.566926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566904    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.567002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.566970    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d314cad3-3324-480b-81f6-9e102076a008-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.567073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.567056    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.567175 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.567126    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-openshift-ca" pod "b8520185-8f72-472c-bc54-a8dfb09da2db" (UID: "b8520185-8f72-472c-bc54-a8dfb09da2db"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:24.567388 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.567255    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.567506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.567489    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.567809 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.567763    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.568611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.567878    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.568611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.567911    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d314cad3-3324-480b-81f6-9e102076a008-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.568611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.567939    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc8kf\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-kube-api-access-pc8kf\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.568611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.568060    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:24.568611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.568079    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:24.569344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.569321    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-xks-ca" pod "5347e77d-10d8-49f4-b60f-9e8250af963d" (UID: "5347e77d-10d8-49f4-b60f-9e8250af963d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:24.569451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.569393    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-xks-ca" pod "b8520185-8f72-472c-bc54-a8dfb09da2db" (UID: "b8520185-8f72-472c-bc54-a8dfb09da2db"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:24.569451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.569426    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8520185-8f72-472c-bc54-a8dfb09da2db-kube-api-access-jpvvc" pod "b8520185-8f72-472c-bc54-a8dfb09da2db" (UID: "b8520185-8f72-472c-bc54-a8dfb09da2db"). InnerVolumeSpecName "kube-api-access-jpvvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:24.569557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.569460    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5347e77d-10d8-49f4-b60f-9e8250af963d-kube-api-access-c2jbv" pod "5347e77d-10d8-49f4-b60f-9e8250af963d" (UID: "5347e77d-10d8-49f4-b60f-9e8250af963d"). InnerVolumeSpecName "kube-api-access-c2jbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:24.571575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.571552    5786 scope.go:122] "RemoveContainer" containerID="a687bd023d0563ed11b6d749122157cb07e80f3d26217292c2b5596bedd90e49"
Oct 30 21:34:24.597686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.597667    5786 scope.go:122] "RemoveContainer" containerID="6e0810cd32da337fbb45fc135b5f170122ae65de9ead0af6241aac8fc2779d27"
Oct 30 21:34:24.669579 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669553    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.669686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669602    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.669686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669660    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.669686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669680    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.669818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669693    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d314cad3-3324-480b-81f6-9e102076a008-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.669818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669717    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc8kf\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-kube-api-access-pc8kf\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.669930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669848    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.669930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669899    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d314cad3-3324-480b-81f6-9e102076a008-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.670033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669968    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.670033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.669987    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-data\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.670131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.670046    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpvvc\" (UniqueName: \"kubernetes.io/projected/b8520185-8f72-472c-bc54-a8dfb09da2db-kube-api-access-jpvvc\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:24.670131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.670062    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2jbv\" (UniqueName: \"kubernetes.io/projected/5347e77d-10d8-49f4-b60f-9e8250af963d-kube-api-access-c2jbv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:24.670131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.670071    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-certs\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.670131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.670079    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5347e77d-10d8-49f4-b60f-9e8250af963d-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:24.670131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.670127    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8520185-8f72-472c-bc54-a8dfb09da2db-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:24.670350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.670143    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.670350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.670307    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-credential-socket\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.670831 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.670809    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d314cad3-3324-480b-81f6-9e102076a008-istiod-ca-cert\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.671899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.671883    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-envoy\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.672116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.672101    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d314cad3-3324-480b-81f6-9e102076a008-istio-podinfo\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.726493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.726465    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc8kf\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-kube-api-access-pc8kf\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.726649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.726472    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-istio-token\") pod \"e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") " pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:24.781380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:24.781313    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:25.051953 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.051874    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"]
Oct 30 21:34:25.055161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.055133    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0abc5437-8kdbd"]
Oct 30 21:34:25.170179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.170147    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"]
Oct 30 21:34:25.171024 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:25.171000    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd314cad3_3324_480b_81f6_9e102076a008.slice/crio-34525a25651ebf8645bf15cd4af76abb5bc48f891dfc9d9ea31e8c95914d3cde WatchSource:0}: Error finding container 34525a25651ebf8645bf15cd4af76abb5bc48f891dfc9d9ea31e8c95914d3cde: Status 404 returned error can't find the container with id 34525a25651ebf8645bf15cd4af76abb5bc48f891dfc9d9ea31e8c95914d3cde
Oct 30 21:34:25.174005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.173975    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:25.174090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.174037    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:25.174090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.174063    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:25.230054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.230030    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" event={"ID":"d314cad3-3324-480b-81f6-9e102076a008","Type":"ContainerStarted","Data":"34525a25651ebf8645bf15cd4af76abb5bc48f891dfc9d9ea31e8c95914d3cde"}
Oct 30 21:34:25.231283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.231269    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"
Oct 30 21:34:25.231365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.231297    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc" event={"ID":"b8520185-8f72-472c-bc54-a8dfb09da2db","Type":"ContainerDied","Data":"c832c612f0e22d1ba8e8b436a9bf2ccd343c09e183520245e42a94ed7a1902a5"}
Oct 30 21:34:25.231365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.231329    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c832c612f0e22d1ba8e8b436a9bf2ccd343c09e183520245e42a94ed7a1902a5"
Oct 30 21:34:25.232631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.232613    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds" event={"ID":"5347e77d-10d8-49f4-b60f-9e8250af963d","Type":"ContainerDied","Data":"4bb9268646bf5800143e8803006fc732b5b9ca90dae0e2881990bec1ec9cf5cc"}
Oct 30 21:34:25.232631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.232629    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"
Oct 30 21:34:25.232751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:25.232651    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bb9268646bf5800143e8803006fc732b5b9ca90dae0e2881990bec1ec9cf5cc"
Oct 30 21:34:26.241100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:26.241063    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" event={"ID":"d314cad3-3324-480b-81f6-9e102076a008","Type":"ContainerStarted","Data":"c9f73d7c1bf21c7c122ce97f6434a36473828be0c9f373be01f863b66f6ada85"}
Oct 30 21:34:26.363232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:26.361100    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podStartSLOduration=2.361081364 podStartE2EDuration="2.361081364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:26.358667419 +0000 UTC m=+1322.229786551" watchObservedRunningTime="2026-10-30 21:34:26.361081364 +0000 UTC m=+1322.232200487"
Oct 30 21:34:26.383220 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:26.383181    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7a24106-afed-4372-ab3b-d9ea2773bf63" path="/var/lib/kubelet/pods/e7a24106-afed-4372-ab3b-d9ea2773bf63/volumes"
Oct 30 21:34:26.786398 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:26.786365    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:26.787987 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:26.787938    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:34:26.788119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:26.788008    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:34:27.782512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:27.782480    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:34:27.782882 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:27.782547    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:34:28.782536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:28.782505    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:34:28.783025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:28.782556    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:34:29.069715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:29.069618    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"]
Oct 30 21:34:29.123958 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:29.123922    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-944bb9ec-r5tbv"]
Oct 30 21:34:29.782603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:29.782570    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:34:29.783098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:29.782629    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:34:30.384279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:30.384249    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b2d7a3b-acaa-4721-975e-7c945cf77345" path="/var/lib/kubelet/pods/0b2d7a3b-acaa-4721-975e-7c945cf77345/volumes"
Oct 30 21:34:30.782541 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:30.782512    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:34:30.782743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:30.782563    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:34:31.782362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:31.782328    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:34:31.782542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:31.782391    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:34:32.786675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:32.786647    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:32.787065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:32.786987    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:32.787806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:32.787786    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:34:39.112923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.112891    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"]
Oct 30 21:34:39.113510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.113496    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5347e77d-10d8-49f4-b60f-9e8250af963d" containerName="revoke-keys"
Oct 30 21:34:39.113557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.113511    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5347e77d-10d8-49f4-b60f-9e8250af963d" containerName="revoke-keys"
Oct 30 21:34:39.113557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.113520    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b8520185-8f72-472c-bc54-a8dfb09da2db" containerName="revoke-keys"
Oct 30 21:34:39.113557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.113524    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8520185-8f72-472c-bc54-a8dfb09da2db" containerName="revoke-keys"
Oct 30 21:34:39.113659 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.113628    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5347e77d-10d8-49f4-b60f-9e8250af963d" containerName="revoke-keys"
Oct 30 21:34:39.113659 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.113637    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8520185-8f72-472c-bc54-a8dfb09da2db" containerName="revoke-keys"
Oct 30 21:34:39.116932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.116911    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.162900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.162873    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"]
Oct 30 21:34:39.166814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.166794    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.177872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.177851    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"]
Oct 30 21:34:39.181958 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.181940    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.182074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.181982    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.182074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182034    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.182186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182076    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.182260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182230    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.182321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182281    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.182321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182312    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.182458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182383    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.182458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182412    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e7280e-7606-4185-913b-9142f7a88f56-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.182458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182443    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4g8g\" (UniqueName: \"kubernetes.io/projected/f997344c-1340-4b42-a44e-1e3493d22100-kube-api-access-b4g8g\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.182913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182470    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e7280e-7606-4185-913b-9142f7a88f56-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.182913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.182524    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j426\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-kube-api-access-7j426\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.220471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.220426    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:34:39.222279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.222253    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"]
Oct 30 21:34:39.224815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.224794    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:34:39.283408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283381    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.283555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283429    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283483    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.283555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283521    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283622    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283703    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283734    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283799    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283828    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e7280e-7606-4185-913b-9142f7a88f56-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283857    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4g8g\" (UniqueName: \"kubernetes.io/projected/f997344c-1340-4b42-a44e-1e3493d22100-kube-api-access-b4g8g\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.283992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283876    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283885    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e7280e-7606-4185-913b-9142f7a88f56-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.283992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.283947    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j426\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-kube-api-access-7j426\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.284493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.284467    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.284674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.284649    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.284862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.284833    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.285138 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.285116    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e7280e-7606-4185-913b-9142f7a88f56-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.285932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.285902    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.286623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.286602    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.288437 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.288409    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.288535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.288491    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e7280e-7606-4185-913b-9142f7a88f56-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.339784 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.339757    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.340343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.340324    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4g8g\" (UniqueName: \"kubernetes.io/projected/f997344c-1340-4b42-a44e-1e3493d22100-kube-api-access-b4g8g\") pod \"maas-api-revoke-sub-36404357-jp2tn\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.340899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.340879    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j426\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-kube-api-access-7j426\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.360804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.360781    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"]
Oct 30 21:34:39.366081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.366036    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.386742 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.386705    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.386912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.386822    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxdw4\" (UniqueName: \"kubernetes.io/projected/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-kube-api-access-gxdw4\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.387090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.387066    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.430825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.430734    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:39.434506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.434477    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"]
Oct 30 21:34:39.487116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.487086    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:39.489314 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.488934    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.489314 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.489036    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxdw4\" (UniqueName: \"kubernetes.io/projected/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-kube-api-access-gxdw4\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.489314 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.489131    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.490041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.490016    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.503482 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.503048    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.547830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.547800    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxdw4\" (UniqueName: \"kubernetes.io/projected/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-kube-api-access-gxdw4\") pod \"maas-api-revoke-sub-c28d39c0-g8bgm\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.687593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.687568    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:39.843193 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.843163    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"]
Oct 30 21:34:39.848383 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.848365    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.886976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.886956    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"]
Oct 30 21:34:39.893911 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:39.893888    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e7280e_7606_4185_913b_9142f7a88f56.slice/crio-c03655ab941eb3baaa9fac05b929418fb861bbdaf3bd145d4e4cd3e704b9bdfc WatchSource:0}: Error finding container c03655ab941eb3baaa9fac05b929418fb861bbdaf3bd145d4e4cd3e704b9bdfc: Status 404 returned error can't find the container with id c03655ab941eb3baaa9fac05b929418fb861bbdaf3bd145d4e4cd3e704b9bdfc
Oct 30 21:34:39.895304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895281    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.895422 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895383    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.895464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895423    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.895499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895489    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.895587 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895569    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.895627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895601    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.895682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895652    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.895719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895693    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.895933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.895916    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnx95\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-kube-api-access-vnx95\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.896165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.896138    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:39.896231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.896197    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:39.896272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.896249    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:39.906582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.906562    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"]
Oct 30 21:34:39.938266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.938193    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"]
Oct 30 21:34:39.943497 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:39.943472    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf997344c_1340_4b42_a44e_1e3493d22100.slice/crio-fe2961379b49df4c64370984a22790ddfe5de3bb5e7bf38d857d2a2d318ef88d WatchSource:0}: Error finding container fe2961379b49df4c64370984a22790ddfe5de3bb5e7bf38d857d2a2d318ef88d: Status 404 returned error can't find the container with id fe2961379b49df4c64370984a22790ddfe5de3bb5e7bf38d857d2a2d318ef88d
Oct 30 21:34:39.997039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997002    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997088    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997161    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnx95\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-kube-api-access-vnx95\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997282    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997356    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997437 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997408    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997452    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997470    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997569    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997647 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997605    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.997839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.997409    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.998417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.998295    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.998417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.998333    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:39.998628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:39.998524    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:40.000368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.000344    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:40.000927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.000900    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:40.054760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.054726    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnx95\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-kube-api-access-vnx95\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:40.054998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.054977    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:40.074332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.074307    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"]
Oct 30 21:34:40.075076 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:40.075048    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca1e3db9_2a03_4bb3_a2bb_8fbc694ae629.slice/crio-d44206d47364ffc458c91f56a21831f4abc6e71bcdbb79192148a26afd9e404b WatchSource:0}: Error finding container d44206d47364ffc458c91f56a21831f4abc6e71bcdbb79192148a26afd9e404b: Status 404 returned error can't find the container with id d44206d47364ffc458c91f56a21831f4abc6e71bcdbb79192148a26afd9e404b
Oct 30 21:34:40.162585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.162562    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:40.472136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.472107    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn" event={"ID":"f997344c-1340-4b42-a44e-1e3493d22100","Type":"ContainerStarted","Data":"fe2961379b49df4c64370984a22790ddfe5de3bb5e7bf38d857d2a2d318ef88d"}
Oct 30 21:34:40.473817 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.473763    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm" event={"ID":"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629","Type":"ContainerStarted","Data":"d44206d47364ffc458c91f56a21831f4abc6e71bcdbb79192148a26afd9e404b"}
Oct 30 21:34:40.475342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.475322    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" event={"ID":"c6e7280e-7606-4185-913b-9142f7a88f56","Type":"ContainerStarted","Data":"92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31"}
Oct 30 21:34:40.475446 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.475353    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" event={"ID":"c6e7280e-7606-4185-913b-9142f7a88f56","Type":"ContainerStarted","Data":"c03655ab941eb3baaa9fac05b929418fb861bbdaf3bd145d4e4cd3e704b9bdfc"}
Oct 30 21:34:40.549484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.549467    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"]
Oct 30 21:34:40.549977 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:40.549957    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf060ad2c_248a_4f3a_a866_1ac2d890bbfd.slice/crio-672402aa82902259b854849bc9f94a976f536765608c19bc9703866ef70433ea WatchSource:0}: Error finding container 672402aa82902259b854849bc9f94a976f536765608c19bc9703866ef70433ea: Status 404 returned error can't find the container with id 672402aa82902259b854849bc9f94a976f536765608c19bc9703866ef70433ea
Oct 30 21:34:40.552548 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.552520    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:40.552626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.552589    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:40.552668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.552625    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:40.587917 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.587867    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn" podStartSLOduration=1.587854402 podStartE2EDuration="1.587854402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:40.586124294 +0000 UTC m=+1336.457243415" watchObservedRunningTime="2026-10-30 21:34:40.587854402 +0000 UTC m=+1336.458973553"
Oct 30 21:34:40.704244 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.701421    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm" podStartSLOduration=1.7014055620000001 podStartE2EDuration="1.701405562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:40.699388314 +0000 UTC m=+1336.570507434" watchObservedRunningTime="2026-10-30 21:34:40.701405562 +0000 UTC m=+1336.572524683"
Oct 30 21:34:40.817692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:40.817578    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podStartSLOduration=1.817559374 podStartE2EDuration="1.817559374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:40.816073293 +0000 UTC m=+1336.687192410" watchObservedRunningTime="2026-10-30 21:34:40.817559374 +0000 UTC m=+1336.688678496"
Oct 30 21:34:41.057485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.051834    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"]
Oct 30 21:34:41.084077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.084044    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-823f4d5f-fsf8q"]
Oct 30 21:34:41.431813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.431781    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:41.433484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.433459    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:34:41.433607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.433510    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 21:34:41.508664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.505638    5786 generic.go:356] "Generic (PLEG): container finished" podID="ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629" containerID="31a810d433bcf64b1b073b954dea4377a71238adb145a205696c8da925fb20d1" exitCode=0
Oct 30 21:34:41.508664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.506171    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm" event={"ID":"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629","Type":"ContainerDied","Data":"31a810d433bcf64b1b073b954dea4377a71238adb145a205696c8da925fb20d1"}
Oct 30 21:34:41.508664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.508304    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" event={"ID":"f060ad2c-248a-4f3a-a866-1ac2d890bbfd","Type":"ContainerStarted","Data":"6c1de686e80d399060017c1a4ac7d6003eedbff0e6d4845ce6a4a69865c79746"}
Oct 30 21:34:41.508664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.508350    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" event={"ID":"f060ad2c-248a-4f3a-a866-1ac2d890bbfd","Type":"ContainerStarted","Data":"672402aa82902259b854849bc9f94a976f536765608c19bc9703866ef70433ea"}
Oct 30 21:34:41.510101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.509973    5786 generic.go:356] "Generic (PLEG): container finished" podID="f997344c-1340-4b42-a44e-1e3493d22100" containerID="7c32302b4ef0503aed324f60f2bdd60a1736e22d5303f5c1992d6dd8f2db3bb8" exitCode=0
Oct 30 21:34:41.510101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.510053    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn" event={"ID":"f997344c-1340-4b42-a44e-1e3493d22100","Type":"ContainerDied","Data":"7c32302b4ef0503aed324f60f2bdd60a1736e22d5303f5c1992d6dd8f2db3bb8"}
Oct 30 21:34:41.901323 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:41.900985    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" podStartSLOduration=2.900964213 podStartE2EDuration="2.900964213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:41.894934858 +0000 UTC m=+1337.766053978" watchObservedRunningTime="2026-10-30 21:34:41.900964213 +0000 UTC m=+1337.772083336"
Oct 30 21:34:42.168227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.164527    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:42.176229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.172318    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:34:42.176229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.172377    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 21:34:42.408348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.408318    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="928bd384-fd3f-4e0a-80b9-201ee5a2a781" path="/var/lib/kubelet/pods/928bd384-fd3f-4e0a-80b9-201ee5a2a781/volumes"
Oct 30 21:34:42.431908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.431842    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:34:42.431908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.431888    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 21:34:42.914166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.914139    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:42.918863 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.918840    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:42.939487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.939454    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-kube-api-access-gxdw4\" (UniqueName: \"kubernetes.io/projected/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-kube-api-access-gxdw4\") pod \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") "
Oct 30 21:34:42.939603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.939519    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-xks-ca\") pod \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") "
Oct 30 21:34:42.939680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.939667    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-openshift-ca\") pod \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\" (UID: \"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629\") "
Oct 30 21:34:42.939974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.939952    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-openshift-ca" pod "ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629" (UID: "ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:42.940116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.940100    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:42.941656 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.941635    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-xks-ca" pod "ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629" (UID: "ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:42.941724 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:42.941690    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-kube-api-access-gxdw4" pod "ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629" (UID: "ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629"). InnerVolumeSpecName "kube-api-access-gxdw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:43.041270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.040593    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f997344c-1340-4b42-a44e-1e3493d22100-kube-api-access-b4g8g\" (UniqueName: \"kubernetes.io/projected/f997344c-1340-4b42-a44e-1e3493d22100-kube-api-access-b4g8g\") pod \"f997344c-1340-4b42-a44e-1e3493d22100\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") "
Oct 30 21:34:43.041270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.040720    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-xks-ca\") pod \"f997344c-1340-4b42-a44e-1e3493d22100\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") "
Oct 30 21:34:43.041270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.040754    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-openshift-ca\") pod \"f997344c-1340-4b42-a44e-1e3493d22100\" (UID: \"f997344c-1340-4b42-a44e-1e3493d22100\") "
Oct 30 21:34:43.041270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.041057    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxdw4\" (UniqueName: \"kubernetes.io/projected/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-kube-api-access-gxdw4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:43.041270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.041068    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:43.041672 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.041414    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-openshift-ca" pod "f997344c-1340-4b42-a44e-1e3493d22100" (UID: "f997344c-1340-4b42-a44e-1e3493d22100"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:43.044126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.044106    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-xks-ca" pod "f997344c-1340-4b42-a44e-1e3493d22100" (UID: "f997344c-1340-4b42-a44e-1e3493d22100"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:43.044350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.044323    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f997344c-1340-4b42-a44e-1e3493d22100-kube-api-access-b4g8g" pod "f997344c-1340-4b42-a44e-1e3493d22100" (UID: "f997344c-1340-4b42-a44e-1e3493d22100"). InnerVolumeSpecName "kube-api-access-b4g8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:43.142780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.142749    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:43.142780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.142776    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f997344c-1340-4b42-a44e-1e3493d22100-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:43.142780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.142785    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4g8g\" (UniqueName: \"kubernetes.io/projected/f997344c-1340-4b42-a44e-1e3493d22100-kube-api-access-b4g8g\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:43.174230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.167545    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:34:43.174230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.167600    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 21:34:43.433699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.433666    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:34:43.435785 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.435287    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 21:34:43.505233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.499943    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"]
Oct 30 21:34:43.515228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.512118    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629" containerName="revoke-keys"
Oct 30 21:34:43.515228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.512147    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629" containerName="revoke-keys"
Oct 30 21:34:43.515228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.512245    5786 cpu_manager.go:414] "containerMap: removing container" podUID="f997344c-1340-4b42-a44e-1e3493d22100" containerName="revoke-keys"
Oct 30 21:34:43.515228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.512253    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f997344c-1340-4b42-a44e-1e3493d22100" containerName="revoke-keys"
Oct 30 21:34:43.515228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.512425    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="f997344c-1340-4b42-a44e-1e3493d22100" containerName="revoke-keys"
Oct 30 21:34:43.515228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.512445    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629" containerName="revoke-keys"
Oct 30 21:34:43.531260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.531161    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.543762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.543735    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"
Oct 30 21:34:43.543896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.543769    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn" event={"ID":"f997344c-1340-4b42-a44e-1e3493d22100","Type":"ContainerDied","Data":"fe2961379b49df4c64370984a22790ddfe5de3bb5e7bf38d857d2a2d318ef88d"}
Oct 30 21:34:43.543896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.543815    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe2961379b49df4c64370984a22790ddfe5de3bb5e7bf38d857d2a2d318ef88d"
Oct 30 21:34:43.545483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.545397    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"
Oct 30 21:34:43.545483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.545410    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm" event={"ID":"ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629","Type":"ContainerDied","Data":"d44206d47364ffc458c91f56a21831f4abc6e71bcdbb79192148a26afd9e404b"}
Oct 30 21:34:43.545483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.545459    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d44206d47364ffc458c91f56a21831f4abc6e71bcdbb79192148a26afd9e404b"
Oct 30 21:34:43.610362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.610328    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"]
Oct 30 21:34:43.667127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.667095    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.672441 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.672417    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xdnk\" (UniqueName: \"kubernetes.io/projected/91be25e6-e157-4623-98c7-a3da3eaef22f-kube-api-access-7xdnk\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.680238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.672791    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.794274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.782334    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.794274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.782409    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdnk\" (UniqueName: \"kubernetes.io/projected/91be25e6-e157-4623-98c7-a3da3eaef22f-kube-api-access-7xdnk\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.794274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.782830    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.794274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.784034    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.794274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.793648    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.842694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.842665    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdnk\" (UniqueName: \"kubernetes.io/projected/91be25e6-e157-4623-98c7-a3da3eaef22f-kube-api-access-7xdnk\") pod \"maas-api-revoke-sub-a8f87036-6x7tm\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:43.896350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:43.896318    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:44.113344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.112806    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"]
Oct 30 21:34:44.119279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.119252    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.137028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.137006    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"]
Oct 30 21:34:44.204610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.204084    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz97v\" (UniqueName: \"kubernetes.io/projected/74dac01f-a3e2-41f9-a7d1-7acde185faca-kube-api-access-mz97v\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.204610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.204234    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.204610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.204421    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.206557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.206524    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:34:44.206700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.206684    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 21:34:44.305439 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.305397    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz97v\" (UniqueName: \"kubernetes.io/projected/74dac01f-a3e2-41f9-a7d1-7acde185faca-kube-api-access-mz97v\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.306036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.306015    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.306933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.306916    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.307941 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.307649    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.308778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.308760    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"]
Oct 30 21:34:44.309499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.309483    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.310116 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:44.310095    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91be25e6_e157_4623_98c7_a3da3eaef22f.slice/crio-142d67abeadc13dddbba9ca6e778ea5e016553367cafa5160e0dbc4064a69399 WatchSource:0}: Error finding container 142d67abeadc13dddbba9ca6e778ea5e016553367cafa5160e0dbc4064a69399: Status 404 returned error can't find the container with id 142d67abeadc13dddbba9ca6e778ea5e016553367cafa5160e0dbc4064a69399
Oct 30 21:34:44.432066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.432033    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:34:44.432241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.432089    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 21:34:44.475230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.467914    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz97v\" (UniqueName: \"kubernetes.io/projected/74dac01f-a3e2-41f9-a7d1-7acde185faca-kube-api-access-mz97v\") pod \"maas-api-revoke-sub-2c9dc34f-4gnxg\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:44.603271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.597132    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm" event={"ID":"91be25e6-e157-4623-98c7-a3da3eaef22f","Type":"ContainerStarted","Data":"142d67abeadc13dddbba9ca6e778ea5e016553367cafa5160e0dbc4064a69399"}
Oct 30 21:34:44.734235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:44.734195    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:45.055421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.055274    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"]
Oct 30 21:34:45.107101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.107065    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e1546fd-h8brs"]
Oct 30 21:34:45.174581 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.174549    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:34:45.175557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.175522    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 21:34:45.208054 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:45.208020    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74dac01f_a3e2_41f9_a7d1_7acde185faca.slice/crio-ebf233f8333c959f1eec844defd7f49644ae22c12d1a55ca751747d51225bfe9 WatchSource:0}: Error finding container ebf233f8333c959f1eec844defd7f49644ae22c12d1a55ca751747d51225bfe9: Status 404 returned error can't find the container with id ebf233f8333c959f1eec844defd7f49644ae22c12d1a55ca751747d51225bfe9
Oct 30 21:34:45.210018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.209990    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"]
Oct 30 21:34:45.432333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.432302    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:34:45.432525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.432358    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 21:34:45.612132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.612053    5786 generic.go:356] "Generic (PLEG): container finished" podID="91be25e6-e157-4623-98c7-a3da3eaef22f" containerID="097ebec7f1ef1d2e34ffb2440b34bdd17b6534670dbb2bd92bd3605b629d062f" exitCode=0
Oct 30 21:34:45.612586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.612367    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm" event={"ID":"91be25e6-e157-4623-98c7-a3da3eaef22f","Type":"ContainerDied","Data":"097ebec7f1ef1d2e34ffb2440b34bdd17b6534670dbb2bd92bd3605b629d062f"}
Oct 30 21:34:45.614285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:45.614242    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg" event={"ID":"74dac01f-a3e2-41f9-a7d1-7acde185faca","Type":"ContainerStarted","Data":"ebf233f8333c959f1eec844defd7f49644ae22c12d1a55ca751747d51225bfe9"}
Oct 30 21:34:46.169363 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.163548    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:34:46.169363 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.163607    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 21:34:46.384291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.384259    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9b6c27c-6a3b-4429-8de8-26d2f4181697" path="/var/lib/kubelet/pods/a9b6c27c-6a3b-4429-8de8-26d2f4181697/volumes"
Oct 30 21:34:46.431624 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.431560    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:34:46.431624 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.431606    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 21:34:46.633407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.633372    5786 generic.go:356] "Generic (PLEG): container finished" podID="74dac01f-a3e2-41f9-a7d1-7acde185faca" containerID="9b4832b48c578a90e19b8c4ad71c114d1b57d15d66157d1a5188a9336a8cc046" exitCode=0
Oct 30 21:34:46.633811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.633459    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg" event={"ID":"74dac01f-a3e2-41f9-a7d1-7acde185faca","Type":"ContainerDied","Data":"9b4832b48c578a90e19b8c4ad71c114d1b57d15d66157d1a5188a9336a8cc046"}
Oct 30 21:34:46.826326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.826300    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:46.943485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.943457    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-xks-ca\") pod \"91be25e6-e157-4623-98c7-a3da3eaef22f\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") "
Oct 30 21:34:46.943754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.943734    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91be25e6-e157-4623-98c7-a3da3eaef22f-kube-api-access-7xdnk\" (UniqueName: \"kubernetes.io/projected/91be25e6-e157-4623-98c7-a3da3eaef22f-kube-api-access-7xdnk\") pod \"91be25e6-e157-4623-98c7-a3da3eaef22f\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") "
Oct 30 21:34:46.943910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.943894    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-openshift-ca\") pod \"91be25e6-e157-4623-98c7-a3da3eaef22f\" (UID: \"91be25e6-e157-4623-98c7-a3da3eaef22f\") "
Oct 30 21:34:46.944744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.944715    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-openshift-ca" pod "91be25e6-e157-4623-98c7-a3da3eaef22f" (UID: "91be25e6-e157-4623-98c7-a3da3eaef22f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:46.948347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.948319    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-xks-ca" pod "91be25e6-e157-4623-98c7-a3da3eaef22f" (UID: "91be25e6-e157-4623-98c7-a3da3eaef22f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:46.948531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:46.948504    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91be25e6-e157-4623-98c7-a3da3eaef22f-kube-api-access-7xdnk" pod "91be25e6-e157-4623-98c7-a3da3eaef22f" (UID: "91be25e6-e157-4623-98c7-a3da3eaef22f"). InnerVolumeSpecName "kube-api-access-7xdnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:47.045862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.045778    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xdnk\" (UniqueName: \"kubernetes.io/projected/91be25e6-e157-4623-98c7-a3da3eaef22f-kube-api-access-7xdnk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:47.045862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.045809    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:47.045862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.045822    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91be25e6-e157-4623-98c7-a3da3eaef22f-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:47.163359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.163325    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:34:47.163503 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.163388    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 21:34:47.435746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.435720    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:47.436161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.436141    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:47.436775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.436755    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:34:47.639596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.639560    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"]
Oct 30 21:34:47.640570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.640546    5786 cpu_manager.go:414] "containerMap: removing container" podUID="91be25e6-e157-4623-98c7-a3da3eaef22f" containerName="revoke-keys"
Oct 30 21:34:47.640673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.640572    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91be25e6-e157-4623-98c7-a3da3eaef22f" containerName="revoke-keys"
Oct 30 21:34:47.640763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.640749    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="91be25e6-e157-4623-98c7-a3da3eaef22f" containerName="revoke-keys"
Oct 30 21:34:47.642153 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.642129    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"
Oct 30 21:34:47.645066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.645023    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm" event={"ID":"91be25e6-e157-4623-98c7-a3da3eaef22f","Type":"ContainerDied","Data":"142d67abeadc13dddbba9ca6e778ea5e016553367cafa5160e0dbc4064a69399"}
Oct 30 21:34:47.645066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.645068    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="142d67abeadc13dddbba9ca6e778ea5e016553367cafa5160e0dbc4064a69399"
Oct 30 21:34:47.645305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.645139    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.715681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.715608    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"]
Oct 30 21:34:47.759814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.758516    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.759814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.758571    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.759814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.758810    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp6h8\" (UniqueName: \"kubernetes.io/projected/29b66767-b1f7-45ca-a5d4-88be10b6aeea-kube-api-access-xp6h8\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.862285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.860249    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.862285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.860298    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.862285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.860481    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp6h8\" (UniqueName: \"kubernetes.io/projected/29b66767-b1f7-45ca-a5d4-88be10b6aeea-kube-api-access-xp6h8\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.862285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.861482    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.865437 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.865380    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.898952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.898926    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:47.916667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.916645    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp6h8\" (UniqueName: \"kubernetes.io/projected/29b66767-b1f7-45ca-a5d4-88be10b6aeea-kube-api-access-xp6h8\") pod \"maas-api-revoke-sub-5953ef0b-wlwls\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:47.961218 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.961176    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-openshift-ca\") pod \"74dac01f-a3e2-41f9-a7d1-7acde185faca\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") "
Oct 30 21:34:47.961392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.961373    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-xks-ca\") pod \"74dac01f-a3e2-41f9-a7d1-7acde185faca\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") "
Oct 30 21:34:47.961605 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.961581    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-openshift-ca" pod "74dac01f-a3e2-41f9-a7d1-7acde185faca" (UID: "74dac01f-a3e2-41f9-a7d1-7acde185faca"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:47.961678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.961626    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74dac01f-a3e2-41f9-a7d1-7acde185faca-kube-api-access-mz97v\" (UniqueName: \"kubernetes.io/projected/74dac01f-a3e2-41f9-a7d1-7acde185faca-kube-api-access-mz97v\") pod \"74dac01f-a3e2-41f9-a7d1-7acde185faca\" (UID: \"74dac01f-a3e2-41f9-a7d1-7acde185faca\") "
Oct 30 21:34:47.962218 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.962185    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:47.963486 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.963455    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-xks-ca" pod "74dac01f-a3e2-41f9-a7d1-7acde185faca" (UID: "74dac01f-a3e2-41f9-a7d1-7acde185faca"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:47.963786 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.963765    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74dac01f-a3e2-41f9-a7d1-7acde185faca-kube-api-access-mz97v" pod "74dac01f-a3e2-41f9-a7d1-7acde185faca" (UID: "74dac01f-a3e2-41f9-a7d1-7acde185faca"). InnerVolumeSpecName "kube-api-access-mz97v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:47.967927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:47.967875    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:48.066950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.063350    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz97v\" (UniqueName: \"kubernetes.io/projected/74dac01f-a3e2-41f9-a7d1-7acde185faca-kube-api-access-mz97v\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:48.066950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.063387    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74dac01f-a3e2-41f9-a7d1-7acde185faca-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:48.167415 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.167387    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:48.167843 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.167826    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:48.168613 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.168593    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:34:48.387278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.381687    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"]
Oct 30 21:34:48.652331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.652243    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"
Oct 30 21:34:48.652760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.652243    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg" event={"ID":"74dac01f-a3e2-41f9-a7d1-7acde185faca","Type":"ContainerDied","Data":"ebf233f8333c959f1eec844defd7f49644ae22c12d1a55ca751747d51225bfe9"}
Oct 30 21:34:48.652760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.652368    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebf233f8333c959f1eec844defd7f49644ae22c12d1a55ca751747d51225bfe9"
Oct 30 21:34:48.653981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:48.653957    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls" event={"ID":"29b66767-b1f7-45ca-a5d4-88be10b6aeea","Type":"ContainerStarted","Data":"1658f9966d5222aec2f8bc65345f01e0fa0a32d5f23b6849c1b250774ddad5b5"}
Oct 30 21:34:49.663058 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.662984    5786 generic.go:356] "Generic (PLEG): container finished" podID="29b66767-b1f7-45ca-a5d4-88be10b6aeea" containerID="92879e6f1ef287fb38c87614f62668df1f742bca10bf601bcc58e6dd18d41af0" exitCode=0
Oct 30 21:34:49.663058 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.663045    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls" event={"ID":"29b66767-b1f7-45ca-a5d4-88be10b6aeea","Type":"ContainerDied","Data":"92879e6f1ef287fb38c87614f62668df1f742bca10bf601bcc58e6dd18d41af0"}
Oct 30 21:34:49.872675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.872648    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"]
Oct 30 21:34:49.873243 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.873228    5786 cpu_manager.go:414] "containerMap: removing container" podUID="74dac01f-a3e2-41f9-a7d1-7acde185faca" containerName="revoke-keys"
Oct 30 21:34:49.873296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.873244    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74dac01f-a3e2-41f9-a7d1-7acde185faca" containerName="revoke-keys"
Oct 30 21:34:49.873393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.873384    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="74dac01f-a3e2-41f9-a7d1-7acde185faca" containerName="revoke-keys"
Oct 30 21:34:49.876644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.876630    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:49.932570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.932544    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"]
Oct 30 21:34:49.996222 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.996176    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:49.996350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.996280    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vrvf\" (UniqueName: \"kubernetes.io/projected/d3b3704b-94f6-4fae-bbb4-a0de88973316-kube-api-access-9vrvf\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:49.996424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:49.996396    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:50.098030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.098005    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:50.098145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.098120    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:50.098145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.098143    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vrvf\" (UniqueName: \"kubernetes.io/projected/d3b3704b-94f6-4fae-bbb4-a0de88973316-kube-api-access-9vrvf\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:50.098767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.098751    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:50.100430 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.100415    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:50.154097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.154078    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vrvf\" (UniqueName: \"kubernetes.io/projected/d3b3704b-94f6-4fae-bbb4-a0de88973316-kube-api-access-9vrvf\") pod \"maas-api-revoke-sub-7cc2a2ae-6g4m2\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:50.190094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.190047    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:50.565252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.565230    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"]
Oct 30 21:34:50.565701 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:50.565674    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b3704b_94f6_4fae_bbb4_a0de88973316.slice/crio-1b86229b5226aace324540582b110a42ddbe8ce485791586eab5de5788c3f9b2 WatchSource:0}: Error finding container 1b86229b5226aace324540582b110a42ddbe8ce485791586eab5de5788c3f9b2: Status 404 returned error can't find the container with id 1b86229b5226aace324540582b110a42ddbe8ce485791586eab5de5788c3f9b2
Oct 30 21:34:50.671703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.671670    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2" event={"ID":"d3b3704b-94f6-4fae-bbb4-a0de88973316","Type":"ContainerStarted","Data":"1b86229b5226aace324540582b110a42ddbe8ce485791586eab5de5788c3f9b2"}
Oct 30 21:34:50.825641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.825617    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:50.909122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.909101    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-openshift-ca\") pod \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") "
Oct 30 21:34:50.909268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.909252    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-xks-ca\") pod \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") "
Oct 30 21:34:50.909432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.909417    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29b66767-b1f7-45ca-a5d4-88be10b6aeea-kube-api-access-xp6h8\" (UniqueName: \"kubernetes.io/projected/29b66767-b1f7-45ca-a5d4-88be10b6aeea-kube-api-access-xp6h8\") pod \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\" (UID: \"29b66767-b1f7-45ca-a5d4-88be10b6aeea\") "
Oct 30 21:34:50.909481 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.909455    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-openshift-ca" pod "29b66767-b1f7-45ca-a5d4-88be10b6aeea" (UID: "29b66767-b1f7-45ca-a5d4-88be10b6aeea"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:50.909818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.909801    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:50.911155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.911140    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-xks-ca" pod "29b66767-b1f7-45ca-a5d4-88be10b6aeea" (UID: "29b66767-b1f7-45ca-a5d4-88be10b6aeea"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:50.911346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:50.911325    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29b66767-b1f7-45ca-a5d4-88be10b6aeea-kube-api-access-xp6h8" pod "29b66767-b1f7-45ca-a5d4-88be10b6aeea" (UID: "29b66767-b1f7-45ca-a5d4-88be10b6aeea"). InnerVolumeSpecName "kube-api-access-xp6h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:51.011791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:51.011766    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29b66767-b1f7-45ca-a5d4-88be10b6aeea-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:51.011791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:51.011791    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp6h8\" (UniqueName: \"kubernetes.io/projected/29b66767-b1f7-45ca-a5d4-88be10b6aeea-kube-api-access-xp6h8\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:51.681494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:51.681425    5786 generic.go:356] "Generic (PLEG): container finished" podID="d3b3704b-94f6-4fae-bbb4-a0de88973316" containerID="d9d972c8c0033fc4ac5f95447ce7ce468cdfe80b05d7d8291744989ded7d38fb" exitCode=0
Oct 30 21:34:51.681494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:51.681475    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2" event={"ID":"d3b3704b-94f6-4fae-bbb4-a0de88973316","Type":"ContainerDied","Data":"d9d972c8c0033fc4ac5f95447ce7ce468cdfe80b05d7d8291744989ded7d38fb"}
Oct 30 21:34:51.683090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:51.683069    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls" event={"ID":"29b66767-b1f7-45ca-a5d4-88be10b6aeea","Type":"ContainerDied","Data":"1658f9966d5222aec2f8bc65345f01e0fa0a32d5f23b6849c1b250774ddad5b5"}
Oct 30 21:34:51.683190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:51.683103    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1658f9966d5222aec2f8bc65345f01e0fa0a32d5f23b6849c1b250774ddad5b5"
Oct 30 21:34:51.683190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:51.683073    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"
Oct 30 21:34:52.362852 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:34:52.362816    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:34:52.835649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:52.835627    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:52.934655 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:52.934627    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-openshift-ca\") pod \"d3b3704b-94f6-4fae-bbb4-a0de88973316\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") "
Oct 30 21:34:52.934807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:52.934792    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-xks-ca\") pod \"d3b3704b-94f6-4fae-bbb4-a0de88973316\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") "
Oct 30 21:34:52.934888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:52.934820    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3b3704b-94f6-4fae-bbb4-a0de88973316-kube-api-access-9vrvf\" (UniqueName: \"kubernetes.io/projected/d3b3704b-94f6-4fae-bbb4-a0de88973316-kube-api-access-9vrvf\") pod \"d3b3704b-94f6-4fae-bbb4-a0de88973316\" (UID: \"d3b3704b-94f6-4fae-bbb4-a0de88973316\") "
Oct 30 21:34:52.934967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:52.934942    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-openshift-ca" pod "d3b3704b-94f6-4fae-bbb4-a0de88973316" (UID: "d3b3704b-94f6-4fae-bbb4-a0de88973316"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:52.935331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:52.935311    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:52.936836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:52.936817    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-xks-ca" pod "d3b3704b-94f6-4fae-bbb4-a0de88973316" (UID: "d3b3704b-94f6-4fae-bbb4-a0de88973316"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:52.937018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:52.936998    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3b3704b-94f6-4fae-bbb4-a0de88973316-kube-api-access-9vrvf" pod "d3b3704b-94f6-4fae-bbb4-a0de88973316" (UID: "d3b3704b-94f6-4fae-bbb4-a0de88973316"). InnerVolumeSpecName "kube-api-access-9vrvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:53.037032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:53.037003    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3b3704b-94f6-4fae-bbb4-a0de88973316-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:53.037032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:53.037030    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vrvf\" (UniqueName: \"kubernetes.io/projected/d3b3704b-94f6-4fae-bbb4-a0de88973316-kube-api-access-9vrvf\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:53.698436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:53.698412    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"
Oct 30 21:34:53.698608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:53.698447    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2" event={"ID":"d3b3704b-94f6-4fae-bbb4-a0de88973316","Type":"ContainerDied","Data":"1b86229b5226aace324540582b110a42ddbe8ce485791586eab5de5788c3f9b2"}
Oct 30 21:34:53.698608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:53.698492    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b86229b5226aace324540582b110a42ddbe8ce485791586eab5de5788c3f9b2"
Oct 30 21:34:54.184050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.184021    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"]
Oct 30 21:34:54.184626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.184612    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d3b3704b-94f6-4fae-bbb4-a0de88973316" containerName="revoke-keys"
Oct 30 21:34:54.184670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.184627    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3b3704b-94f6-4fae-bbb4-a0de88973316" containerName="revoke-keys"
Oct 30 21:34:54.184670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.184656    5786 cpu_manager.go:414] "containerMap: removing container" podUID="29b66767-b1f7-45ca-a5d4-88be10b6aeea" containerName="revoke-keys"
Oct 30 21:34:54.184670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.184661    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29b66767-b1f7-45ca-a5d4-88be10b6aeea" containerName="revoke-keys"
Oct 30 21:34:54.184776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.184767    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3b3704b-94f6-4fae-bbb4-a0de88973316" containerName="revoke-keys"
Oct 30 21:34:54.184808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.184784    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="29b66767-b1f7-45ca-a5d4-88be10b6aeea" containerName="revoke-keys"
Oct 30 21:34:54.188623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.188606    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.241343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.241317    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:34:54.245194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.245167    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"]
Oct 30 21:34:54.289231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.289181    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:34:54.351870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.351840    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.352063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.352041    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdgbg\" (UniqueName: \"kubernetes.io/projected/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-kube-api-access-bdgbg\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.352200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.352185    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.453095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.453039    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdgbg\" (UniqueName: \"kubernetes.io/projected/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-kube-api-access-bdgbg\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.453219 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.453116    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.453219 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.453179    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.453780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.453760    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.455590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.455572    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.509722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.509694    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdgbg\" (UniqueName: \"kubernetes.io/projected/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-kube-api-access-bdgbg\") pod \"maas-api-revoke-sub-1b584c07-5qbjg\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:54.801733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:54.801662    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:55.183016 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.182988    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"]
Oct 30 21:34:55.183781 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:55.183753    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8e4a48_dcc7_40fd_9df4_8db8723c6b5c.slice/crio-8407702c46bb3671640a9abca258636989ae51de706dbb7cd1606b3bd73ee77b WatchSource:0}: Error finding container 8407702c46bb3671640a9abca258636989ae51de706dbb7cd1606b3bd73ee77b: Status 404 returned error can't find the container with id 8407702c46bb3671640a9abca258636989ae51de706dbb7cd1606b3bd73ee77b
Oct 30 21:34:55.242276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.242251    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"]
Oct 30 21:34:55.248259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.248242    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.356259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.346337    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"]
Oct 30 21:34:55.363591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.363566    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8bk2\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-kube-api-access-p8bk2\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.363717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.363605    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.363717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.363636    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.363717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.363654    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.363864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.363847    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.363940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.363924    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.363977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.363953    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.364100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.364081    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.364162 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.364116    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.465479 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.465452    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.465579 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.465510    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.465579 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.465543    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.465694    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.465725    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.465807    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8bk2\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-kube-api-access-p8bk2\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.465840    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.466071    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.466115    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.466147    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.466434    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.466532    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.466544    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.466715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.466533    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.468034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.468016    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.468694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.468678    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.520971 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.520947    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.521460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.521441    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8bk2\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-kube-api-access-p8bk2\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.562022 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.561999    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:55.715706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.715626    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg" event={"ID":"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c","Type":"ContainerStarted","Data":"48a79917a7b51ba7ecadb448f7bd8bc2df5fe297011b2f2af6c11756caea9576"}
Oct 30 21:34:55.715706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.715702    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg" event={"ID":"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c","Type":"ContainerStarted","Data":"8407702c46bb3671640a9abca258636989ae51de706dbb7cd1606b3bd73ee77b"}
Oct 30 21:34:55.828564 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.828514    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg" podStartSLOduration=1.8285009269999999 podStartE2EDuration="1.828500927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:55.827956243 +0000 UTC m=+1351.699075364" watchObservedRunningTime="2026-10-30 21:34:55.828500927 +0000 UTC m=+1351.699620095"
Oct 30 21:34:55.994561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.994537    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"]
Oct 30 21:34:55.995396 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:34:55.995372    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a92f2bb_b279_427b_b8bc_9143dfcd6cff.slice/crio-2e700c0c69f8f70c3abdd4b6f5904979a50f0e2a7138712559a3cf0378eaadfa WatchSource:0}: Error finding container 2e700c0c69f8f70c3abdd4b6f5904979a50f0e2a7138712559a3cf0378eaadfa: Status 404 returned error can't find the container with id 2e700c0c69f8f70c3abdd4b6f5904979a50f0e2a7138712559a3cf0378eaadfa
Oct 30 21:34:55.997833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.997804    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:55.997909 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.997871    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:55.997909 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:55.997899    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:34:56.726955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:56.726868    5786 generic.go:356] "Generic (PLEG): container finished" podID="5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c" containerID="48a79917a7b51ba7ecadb448f7bd8bc2df5fe297011b2f2af6c11756caea9576" exitCode=0
Oct 30 21:34:56.727393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:56.726958    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg" event={"ID":"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c","Type":"ContainerDied","Data":"48a79917a7b51ba7ecadb448f7bd8bc2df5fe297011b2f2af6c11756caea9576"}
Oct 30 21:34:56.728726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:56.728703    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" event={"ID":"1a92f2bb-b279-427b-b8bc-9143dfcd6cff","Type":"ContainerStarted","Data":"64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97"}
Oct 30 21:34:56.728859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:56.728742    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" event={"ID":"1a92f2bb-b279-427b-b8bc-9143dfcd6cff","Type":"ContainerStarted","Data":"2e700c0c69f8f70c3abdd4b6f5904979a50f0e2a7138712559a3cf0378eaadfa"}
Oct 30 21:34:56.978404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:56.978272    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" podStartSLOduration=1.978253855 podStartE2EDuration="1.978253855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:56.972695823 +0000 UTC m=+1352.843814967" watchObservedRunningTime="2026-10-30 21:34:56.978253855 +0000 UTC m=+1352.849372976"
Oct 30 21:34:57.574236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:57.570620    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:34:57.574236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:57.570724    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:34:57.574236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:57.570762    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:34:58.019386 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.019366    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:58.208890 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.208866    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-xks-ca\") pod \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") "
Oct 30 21:34:58.209057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.208955    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-kube-api-access-bdgbg\" (UniqueName: \"kubernetes.io/projected/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-kube-api-access-bdgbg\") pod \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") "
Oct 30 21:34:58.209057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.208991    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-openshift-ca\") pod \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\" (UID: \"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c\") "
Oct 30 21:34:58.209445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.209414    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-openshift-ca" pod "5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c" (UID: "5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:58.209557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.209547    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:58.211097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.211064    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-kube-api-access-bdgbg" pod "5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c" (UID: "5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c"). InnerVolumeSpecName "kube-api-access-bdgbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:58.211201 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.211099    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-xks-ca" pod "5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c" (UID: "5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:58.311257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.311231    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:58.311257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.311258    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdgbg\" (UniqueName: \"kubernetes.io/projected/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c-kube-api-access-bdgbg\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:58.574284 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.573299    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:34:58.574284 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.573352    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:34:58.747528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.747502    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"
Oct 30 21:34:58.747679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.747579    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg" event={"ID":"5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c","Type":"ContainerDied","Data":"8407702c46bb3671640a9abca258636989ae51de706dbb7cd1606b3bd73ee77b"}
Oct 30 21:34:58.747679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:58.747622    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8407702c46bb3671640a9abca258636989ae51de706dbb7cd1606b3bd73ee77b"
Oct 30 21:34:59.568230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.567123    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:34:59.568230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.567189    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:34:59.600329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.600304    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"]
Oct 30 21:34:59.600908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.600896    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c" containerName="revoke-keys"
Oct 30 21:34:59.600948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.600910    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c" containerName="revoke-keys"
Oct 30 21:34:59.601023 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.601014    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c" containerName="revoke-keys"
Oct 30 21:34:59.604518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.604504    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.685316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.682999    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"]
Oct 30 21:34:59.685316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.683387    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:34:59.707978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.707947    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:34:59.729631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.729601    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.729739 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.729664    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc5n8\" (UniqueName: \"kubernetes.io/projected/0f7fae30-d7b9-459b-9040-675b18644d99-kube-api-access-zc5n8\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.729851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.729828    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.830726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.830653    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.830726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.830722    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc5n8\" (UniqueName: \"kubernetes.io/projected/0f7fae30-d7b9-459b-9040-675b18644d99-kube-api-access-zc5n8\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.830899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.830762    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.831488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.831442    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.833444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.833419    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.895135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.895106    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc5n8\" (UniqueName: \"kubernetes.io/projected/0f7fae30-d7b9-459b-9040-675b18644d99-kube-api-access-zc5n8\") pod \"maas-api-revoke-sub-0ade4058-77qxh\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:34:59.917470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:34:59.917445    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:35:00.316044 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:00.316017    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7fae30_d7b9_459b_9040_675b18644d99.slice/crio-75c1248fb95705dfa4c14d8c8aac029793dd3c03efbd691afc437afdacfe869a WatchSource:0}: Error finding container 75c1248fb95705dfa4c14d8c8aac029793dd3c03efbd691afc437afdacfe869a: Status 404 returned error can't find the container with id 75c1248fb95705dfa4c14d8c8aac029793dd3c03efbd691afc437afdacfe869a
Oct 30 21:35:00.316258 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.316234    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"]
Oct 30 21:35:00.571275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.565524    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:35:00.571275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.565573    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:35:00.681713 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.681683    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"]
Oct 30 21:35:00.686167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.686148    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.725085 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.725057    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"]
Oct 30 21:35:00.739144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.739123    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.739290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.739168    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.739358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.739302    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh6zb\" (UniqueName: \"kubernetes.io/projected/670254e0-d1c2-44ec-af01-6ab9897f26fb-kube-api-access-hh6zb\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.799191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.799156    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh" event={"ID":"0f7fae30-d7b9-459b-9040-675b18644d99","Type":"ContainerStarted","Data":"baf7360860900944b800cfbb5d4ec4c31471adb1491829cfd7813e42728c8ecf"}
Oct 30 21:35:00.799389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.799236    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh" event={"ID":"0f7fae30-d7b9-459b-9040-675b18644d99","Type":"ContainerStarted","Data":"75c1248fb95705dfa4c14d8c8aac029793dd3c03efbd691afc437afdacfe869a"}
Oct 30 21:35:00.840708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.840648    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.840823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.840708    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh6zb\" (UniqueName: \"kubernetes.io/projected/670254e0-d1c2-44ec-af01-6ab9897f26fb-kube-api-access-hh6zb\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.841070 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.841049    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.841384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.841367    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.843381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.843365    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.897060 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.897027    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh6zb\" (UniqueName: \"kubernetes.io/projected/670254e0-d1c2-44ec-af01-6ab9897f26fb-kube-api-access-hh6zb\") pod \"maas-api-revoke-sub-f3f62a96-vtxl6\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:00.944104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:00.944055    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh" podStartSLOduration=1.944040636 podStartE2EDuration="1.944040636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:00.939960516 +0000 UTC m=+1356.811079634" watchObservedRunningTime="2026-10-30 21:35:00.944040636 +0000 UTC m=+1356.815159959"
Oct 30 21:35:01.000433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.000407    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:01.060394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.055323    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"]
Oct 30 21:35:01.082298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.082259    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1db764c1-qhsqm"]
Oct 30 21:35:01.398926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.398900    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"]
Oct 30 21:35:01.399380 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:01.399358    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod670254e0_d1c2_44ec_af01_6ab9897f26fb.slice/crio-7ff0cf5b558b60d2876600024143e82ebc7a25e884172715163fa583079c9927 WatchSource:0}: Error finding container 7ff0cf5b558b60d2876600024143e82ebc7a25e884172715163fa583079c9927: Status 404 returned error can't find the container with id 7ff0cf5b558b60d2876600024143e82ebc7a25e884172715163fa583079c9927
Oct 30 21:35:01.576250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.570545    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:35:01.576250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.570602    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:35:01.810644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.810609    5786 generic.go:356] "Generic (PLEG): container finished" podID="0f7fae30-d7b9-459b-9040-675b18644d99" containerID="baf7360860900944b800cfbb5d4ec4c31471adb1491829cfd7813e42728c8ecf" exitCode=0
Oct 30 21:35:01.810820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.810659    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh" event={"ID":"0f7fae30-d7b9-459b-9040-675b18644d99","Type":"ContainerDied","Data":"baf7360860900944b800cfbb5d4ec4c31471adb1491829cfd7813e42728c8ecf"}
Oct 30 21:35:01.812355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.812327    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6" event={"ID":"670254e0-d1c2-44ec-af01-6ab9897f26fb","Type":"ContainerStarted","Data":"ca27ec9e5346ab87d79725a757cd4ee49868998ff6be491f50719bdd11cb9f4f"}
Oct 30 21:35:01.812466 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:01.812377    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6" event={"ID":"670254e0-d1c2-44ec-af01-6ab9897f26fb","Type":"ContainerStarted","Data":"7ff0cf5b558b60d2876600024143e82ebc7a25e884172715163fa583079c9927"}
Oct 30 21:35:02.041064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:02.040958    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6" podStartSLOduration=2.0409412160000002 podStartE2EDuration="2.040941216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:02.039600224 +0000 UTC m=+1357.910719344" watchObservedRunningTime="2026-10-30 21:35:02.040941216 +0000 UTC m=+1357.912060338"
Oct 30 21:35:02.360100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:02.360018    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="770d013e-c334-4993-b248-1c24c7bea1a1" path="/var/lib/kubelet/pods/770d013e-c334-4993-b248-1c24c7bea1a1/volumes"
Oct 30 21:35:02.562411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:02.562388    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:35:02.562549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:02.562431    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:35:02.835014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:02.834976    5786 generic.go:356] "Generic (PLEG): container finished" podID="670254e0-d1c2-44ec-af01-6ab9897f26fb" containerID="ca27ec9e5346ab87d79725a757cd4ee49868998ff6be491f50719bdd11cb9f4f" exitCode=0
Oct 30 21:35:02.835556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:02.835452    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6" event={"ID":"670254e0-d1c2-44ec-af01-6ab9897f26fb","Type":"ContainerDied","Data":"ca27ec9e5346ab87d79725a757cd4ee49868998ff6be491f50719bdd11cb9f4f"}
Oct 30 21:35:03.006493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.006467    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:35:03.071481 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.071452    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-xks-ca\") pod \"0f7fae30-d7b9-459b-9040-675b18644d99\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") "
Oct 30 21:35:03.071639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.071507    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f7fae30-d7b9-459b-9040-675b18644d99-kube-api-access-zc5n8\" (UniqueName: \"kubernetes.io/projected/0f7fae30-d7b9-459b-9040-675b18644d99-kube-api-access-zc5n8\") pod \"0f7fae30-d7b9-459b-9040-675b18644d99\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") "
Oct 30 21:35:03.071639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.071553    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-openshift-ca\") pod \"0f7fae30-d7b9-459b-9040-675b18644d99\" (UID: \"0f7fae30-d7b9-459b-9040-675b18644d99\") "
Oct 30 21:35:03.071938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.071910    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-openshift-ca" pod "0f7fae30-d7b9-459b-9040-675b18644d99" (UID: "0f7fae30-d7b9-459b-9040-675b18644d99"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:03.072119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.072095    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:03.073827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.073807    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-xks-ca" pod "0f7fae30-d7b9-459b-9040-675b18644d99" (UID: "0f7fae30-d7b9-459b-9040-675b18644d99"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:03.074057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.074031    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f7fae30-d7b9-459b-9040-675b18644d99-kube-api-access-zc5n8" pod "0f7fae30-d7b9-459b-9040-675b18644d99" (UID: "0f7fae30-d7b9-459b-9040-675b18644d99"). InnerVolumeSpecName "kube-api-access-zc5n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:03.173061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.173034    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f7fae30-d7b9-459b-9040-675b18644d99-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:03.173061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.173060    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc5n8\" (UniqueName: \"kubernetes.io/projected/0f7fae30-d7b9-459b-9040-675b18644d99-kube-api-access-zc5n8\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:03.567295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.567268    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:35:03.567750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.567724    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:35:03.568483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.568466    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:35:03.847799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.847694    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh" event={"ID":"0f7fae30-d7b9-459b-9040-675b18644d99","Type":"ContainerDied","Data":"75c1248fb95705dfa4c14d8c8aac029793dd3c03efbd691afc437afdacfe869a"}
Oct 30 21:35:03.847799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.847777    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75c1248fb95705dfa4c14d8c8aac029793dd3c03efbd691afc437afdacfe869a"
Oct 30 21:35:03.848390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:03.847839    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"
Oct 30 21:35:04.020630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.020610    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:04.083116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.083088    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/670254e0-d1c2-44ec-af01-6ab9897f26fb-kube-api-access-hh6zb\" (UniqueName: \"kubernetes.io/projected/670254e0-d1c2-44ec-af01-6ab9897f26fb-kube-api-access-hh6zb\") pod \"670254e0-d1c2-44ec-af01-6ab9897f26fb\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") "
Oct 30 21:35:04.083311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.083293    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-xks-ca\") pod \"670254e0-d1c2-44ec-af01-6ab9897f26fb\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") "
Oct 30 21:35:04.083392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.083378    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-openshift-ca\") pod \"670254e0-d1c2-44ec-af01-6ab9897f26fb\" (UID: \"670254e0-d1c2-44ec-af01-6ab9897f26fb\") "
Oct 30 21:35:04.083713 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.083689    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-openshift-ca" pod "670254e0-d1c2-44ec-af01-6ab9897f26fb" (UID: "670254e0-d1c2-44ec-af01-6ab9897f26fb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:04.084150 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.084128    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.085433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.085413    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-xks-ca" pod "670254e0-d1c2-44ec-af01-6ab9897f26fb" (UID: "670254e0-d1c2-44ec-af01-6ab9897f26fb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:04.085515 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.085438    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/670254e0-d1c2-44ec-af01-6ab9897f26fb-kube-api-access-hh6zb" pod "670254e0-d1c2-44ec-af01-6ab9897f26fb" (UID: "670254e0-d1c2-44ec-af01-6ab9897f26fb"). InnerVolumeSpecName "kube-api-access-hh6zb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:04.185653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.185625    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh6zb\" (UniqueName: \"kubernetes.io/projected/670254e0-d1c2-44ec-af01-6ab9897f26fb-kube-api-access-hh6zb\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.185653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.185648    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/670254e0-d1c2-44ec-af01-6ab9897f26fb-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.856065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.856041    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"
Oct 30 21:35:04.856463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.856090    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6" event={"ID":"670254e0-d1c2-44ec-af01-6ab9897f26fb","Type":"ContainerDied","Data":"7ff0cf5b558b60d2876600024143e82ebc7a25e884172715163fa583079c9927"}
Oct 30 21:35:04.856463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.856133    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ff0cf5b558b60d2876600024143e82ebc7a25e884172715163fa583079c9927"
Oct 30 21:35:04.893514 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.893485    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"]
Oct 30 21:35:04.894272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.894253    5786 cpu_manager.go:414] "containerMap: removing container" podUID="670254e0-d1c2-44ec-af01-6ab9897f26fb" containerName="revoke-keys"
Oct 30 21:35:04.894272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.894270    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="670254e0-d1c2-44ec-af01-6ab9897f26fb" containerName="revoke-keys"
Oct 30 21:35:04.894462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.894300    5786 cpu_manager.go:414] "containerMap: removing container" podUID="0f7fae30-d7b9-459b-9040-675b18644d99" containerName="revoke-keys"
Oct 30 21:35:04.894462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.894305    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f7fae30-d7b9-459b-9040-675b18644d99" containerName="revoke-keys"
Oct 30 21:35:04.894462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.894409    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f7fae30-d7b9-459b-9040-675b18644d99" containerName="revoke-keys"
Oct 30 21:35:04.894462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.894424    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="670254e0-d1c2-44ec-af01-6ab9897f26fb" containerName="revoke-keys"
Oct 30 21:35:04.897999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.897983    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:04.950799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.950645    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:35:04.955991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.955964    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"]
Oct 30 21:35:04.959296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.959273    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:35:04.994127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.994102    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:04.994268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.994190    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzrz2\" (UniqueName: \"kubernetes.io/projected/fc32932b-a1ea-4c89-8f51-a171fd161203-kube-api-access-hzrz2\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:04.994344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:04.994325    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:05.095701 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.095670    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:05.095817 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.095781    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:05.095876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.095837    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzrz2\" (UniqueName: \"kubernetes.io/projected/fc32932b-a1ea-4c89-8f51-a171fd161203-kube-api-access-hzrz2\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:05.096431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.096414    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:05.098056 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.098036    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:05.151886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.151830    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzrz2\" (UniqueName: \"kubernetes.io/projected/fc32932b-a1ea-4c89-8f51-a171fd161203-kube-api-access-hzrz2\") pod \"maas-api-revoke-sub-1d44bc68-fgvd9\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:05.212050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.212027    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:05.588059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.588033    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"]
Oct 30 21:35:05.588884 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:05.588854    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc32932b_a1ea_4c89_8f51_a171fd161203.slice/crio-d5a179604916a1e626adeb69c5a329ab76fa585a169e0d7e863f15687dbfc739 WatchSource:0}: Error finding container d5a179604916a1e626adeb69c5a329ab76fa585a169e0d7e863f15687dbfc739: Status 404 returned error can't find the container with id d5a179604916a1e626adeb69c5a329ab76fa585a169e0d7e863f15687dbfc739
Oct 30 21:35:05.866179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:05.866155    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9" event={"ID":"fc32932b-a1ea-4c89-8f51-a171fd161203","Type":"ContainerStarted","Data":"d5a179604916a1e626adeb69c5a329ab76fa585a169e0d7e863f15687dbfc739"}
Oct 30 21:35:06.875264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:06.875228    5786 generic.go:356] "Generic (PLEG): container finished" podID="fc32932b-a1ea-4c89-8f51-a171fd161203" containerID="fd59910f22ca58cb6b2efe10ed90f681217162b81e45a1180c3f3058cf60582b" exitCode=0
Oct 30 21:35:06.875626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:06.875262    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9" event={"ID":"fc32932b-a1ea-4c89-8f51-a171fd161203","Type":"ContainerDied","Data":"fd59910f22ca58cb6b2efe10ed90f681217162b81e45a1180c3f3058cf60582b"}
Oct 30 21:35:08.854231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.853498    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:08.883679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.875875    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-openshift-ca\") pod \"fc32932b-a1ea-4c89-8f51-a171fd161203\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") "
Oct 30 21:35:08.883679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.876193    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-xks-ca\") pod \"fc32932b-a1ea-4c89-8f51-a171fd161203\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") "
Oct 30 21:35:08.883679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.876261    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc32932b-a1ea-4c89-8f51-a171fd161203-kube-api-access-hzrz2\" (UniqueName: \"kubernetes.io/projected/fc32932b-a1ea-4c89-8f51-a171fd161203-kube-api-access-hzrz2\") pod \"fc32932b-a1ea-4c89-8f51-a171fd161203\" (UID: \"fc32932b-a1ea-4c89-8f51-a171fd161203\") "
Oct 30 21:35:08.883679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.883442    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-openshift-ca" pod "fc32932b-a1ea-4c89-8f51-a171fd161203" (UID: "fc32932b-a1ea-4c89-8f51-a171fd161203"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:08.893262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.892710    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-xks-ca" pod "fc32932b-a1ea-4c89-8f51-a171fd161203" (UID: "fc32932b-a1ea-4c89-8f51-a171fd161203"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:08.916436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.899057    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc32932b-a1ea-4c89-8f51-a171fd161203-kube-api-access-hzrz2" pod "fc32932b-a1ea-4c89-8f51-a171fd161203" (UID: "fc32932b-a1ea-4c89-8f51-a171fd161203"). InnerVolumeSpecName "kube-api-access-hzrz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:08.947229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.941396    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9" event={"ID":"fc32932b-a1ea-4c89-8f51-a171fd161203","Type":"ContainerDied","Data":"d5a179604916a1e626adeb69c5a329ab76fa585a169e0d7e863f15687dbfc739"}
Oct 30 21:35:08.947229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.941475    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5a179604916a1e626adeb69c5a329ab76fa585a169e0d7e863f15687dbfc739"
Oct 30 21:35:08.947229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.941564    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"
Oct 30 21:35:09.003228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.995341    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:09.003228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.995375    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzrz2\" (UniqueName: \"kubernetes.io/projected/fc32932b-a1ea-4c89-8f51-a171fd161203-kube-api-access-hzrz2\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:09.003228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:08.995393    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc32932b-a1ea-4c89-8f51-a171fd161203-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:09.363288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:09.362664    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:35:09.452230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:09.443360    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": context deadline exceeded"
Oct 30 21:35:09.494381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:09.494342    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:09.691234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:09.683360    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 21:35:09.691234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:09.687276    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:10.056004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.050114    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": context deadline exceeded"
Oct 30 21:35:10.369581 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.369470    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:10.499902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.499857    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:10.920235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.507652    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:10.920235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.654480    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:10.920235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.654516    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": context deadline exceeded"
Oct 30 21:35:10.920235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.685978    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:10.920235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.691331    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:10.920235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.835107    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:10.920235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.867338    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:35:10.920235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:10.908254    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:11.065234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.060327    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:11.205414 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.205322    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:35:11.208356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.208333    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:35:11.208584 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.208570    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:35:11.212265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.212246    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:35:11.212447 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.212436    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:35:11.298722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.298694    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:35:11.689287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.652353    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/health\": context deadline exceeded"
Oct 30 21:35:11.780419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.729343    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/health\": context deadline exceeded"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.880678    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"]
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.881717    5786 cpu_manager.go:414] "containerMap: removing container" podUID="fc32932b-a1ea-4c89-8f51-a171fd161203" containerName="revoke-keys"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.881739    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc32932b-a1ea-4c89-8f51-a171fd161203" containerName="revoke-keys"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.881900    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc32932b-a1ea-4c89-8f51-a171fd161203" containerName="revoke-keys"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.934336    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.934626    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.934791    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.934933    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.935989    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": context deadline exceeded"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.936171    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.936417    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.936634    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:35:11.941286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:11.936786    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.037299    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.037388    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.037442    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.037476    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.037525    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.037571    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.060133    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.079309    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": context deadline exceeded"
Oct 30 21:35:12.134275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.115883    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:35:12.154247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.153759    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"]
Oct 30 21:35:12.176724 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.176689    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:12.199285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.199182    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.199795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.199772    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29p7x\" (UniqueName: \"kubernetes.io/projected/c42756e3-4b4c-4227-9440-1903037bc148-kube-api-access-29p7x\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.200333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.200311    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.213780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.213744    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/health\": context deadline exceeded"
Oct 30 21:35:12.508290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.508137    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:35:12.575763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.534600    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.575763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.534758    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29p7x\" (UniqueName: \"kubernetes.io/projected/c42756e3-4b4c-4227-9440-1903037bc148-kube-api-access-29p7x\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.575763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.534799    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.628264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.583598    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.628264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.614352    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:12.668013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.662377    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:35:12.745225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.721416    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29p7x\" (UniqueName: \"kubernetes.io/projected/c42756e3-4b4c-4227-9440-1903037bc148-kube-api-access-29p7x\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.883994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.879709    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9be34f0b-kkmtm\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:12.954582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.954530    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:12.954935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.954906    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 30 21:35:12.955187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.955161    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:35:12.955332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.955279    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 21:35:13.183287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.955432    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:35:13.219397 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.955481    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:35:13.221076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.955501    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:35:13.222606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:12.994871    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:13.223520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.139373    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:13.224835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.139578    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:13.295593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.295567    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:35:13.554006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.536943    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": context deadline exceeded"
Oct 30 21:35:13.641703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.641659    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:13.669234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.643474    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:35:13.681444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.681418    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:35:13.799233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.794711    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:35:13.818053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.801318    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:35:13.896388 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.896364    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:35:13.898966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.898948    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:35:13.899143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.899134    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:35:13.899259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.899250    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:35:13.933248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.932678    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:35:13.934261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:13.933718    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:35:14.306062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:14.306037    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"]
Oct 30 21:35:14.307181 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:14.307153    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42756e3_4b4c_4227_9440_1903037bc148.slice/crio-2efe4ef5346947d83d34664ea15edee3658c9f666edb66787d732d5690c56b1d WatchSource:0}: Error finding container 2efe4ef5346947d83d34664ea15edee3658c9f666edb66787d732d5690c56b1d: Status 404 returned error can't find the container with id 2efe4ef5346947d83d34664ea15edee3658c9f666edb66787d732d5690c56b1d
Oct 30 21:35:14.924946 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:14.924913    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm" event={"ID":"c42756e3-4b4c-4227-9440-1903037bc148","Type":"ContainerStarted","Data":"6dd3f36977af90591511a21dca2b0d039059cacf2b1204e9c082bfebbd952741"}
Oct 30 21:35:14.925123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:14.924978    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm" event={"ID":"c42756e3-4b4c-4227-9440-1903037bc148","Type":"ContainerStarted","Data":"2efe4ef5346947d83d34664ea15edee3658c9f666edb66787d732d5690c56b1d"}
Oct 30 21:35:15.274527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:15.274432    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm" podStartSLOduration=4.274418363 podStartE2EDuration="4.274418363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:15.272193222 +0000 UTC m=+1371.143312367" watchObservedRunningTime="2026-10-30 21:35:15.274418363 +0000 UTC m=+1371.145537482"
Oct 30 21:35:15.933770 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:15.933728    5786 generic.go:356] "Generic (PLEG): container finished" podID="c42756e3-4b4c-4227-9440-1903037bc148" containerID="6dd3f36977af90591511a21dca2b0d039059cacf2b1204e9c082bfebbd952741" exitCode=0
Oct 30 21:35:15.934242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:15.933853    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm" event={"ID":"c42756e3-4b4c-4227-9440-1903037bc148","Type":"ContainerDied","Data":"6dd3f36977af90591511a21dca2b0d039059cacf2b1204e9c082bfebbd952741"}
Oct 30 21:35:17.100051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.100026    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:17.124671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.124645    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-xks-ca\") pod \"c42756e3-4b4c-4227-9440-1903037bc148\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") "
Oct 30 21:35:17.124759 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.124700    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-openshift-ca\") pod \"c42756e3-4b4c-4227-9440-1903037bc148\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") "
Oct 30 21:35:17.125026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.125005    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c42756e3-4b4c-4227-9440-1903037bc148-kube-api-access-29p7x\" (UniqueName: \"kubernetes.io/projected/c42756e3-4b4c-4227-9440-1903037bc148-kube-api-access-29p7x\") pod \"c42756e3-4b4c-4227-9440-1903037bc148\" (UID: \"c42756e3-4b4c-4227-9440-1903037bc148\") "
Oct 30 21:35:17.125135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.125008    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-openshift-ca" pod "c42756e3-4b4c-4227-9440-1903037bc148" (UID: "c42756e3-4b4c-4227-9440-1903037bc148"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:17.125481 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.125424    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:17.126792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.126768    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-xks-ca" pod "c42756e3-4b4c-4227-9440-1903037bc148" (UID: "c42756e3-4b4c-4227-9440-1903037bc148"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:17.126867 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.126847    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42756e3-4b4c-4227-9440-1903037bc148-kube-api-access-29p7x" pod "c42756e3-4b4c-4227-9440-1903037bc148" (UID: "c42756e3-4b4c-4227-9440-1903037bc148"). InnerVolumeSpecName "kube-api-access-29p7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:17.235382 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.232789    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29p7x\" (UniqueName: \"kubernetes.io/projected/c42756e3-4b4c-4227-9440-1903037bc148-kube-api-access-29p7x\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:17.235382 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.232833    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c42756e3-4b4c-4227-9440-1903037bc148-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:17.954625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.954596    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"
Oct 30 21:35:17.954813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.954628    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm" event={"ID":"c42756e3-4b4c-4227-9440-1903037bc148","Type":"ContainerDied","Data":"2efe4ef5346947d83d34664ea15edee3658c9f666edb66787d732d5690c56b1d"}
Oct 30 21:35:17.954813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:17.954672    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2efe4ef5346947d83d34664ea15edee3658c9f666edb66787d732d5690c56b1d"
Oct 30 21:35:18.334416 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.334344    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"]
Oct 30 21:35:18.334952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.334939    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c42756e3-4b4c-4227-9440-1903037bc148" containerName="revoke-keys"
Oct 30 21:35:18.335000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.334953    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c42756e3-4b4c-4227-9440-1903037bc148" containerName="revoke-keys"
Oct 30 21:35:18.335070 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.335061    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c42756e3-4b4c-4227-9440-1903037bc148" containerName="revoke-keys"
Oct 30 21:35:18.338652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.338633    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.390439 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.390416    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:35:18.390677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.390637    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:35:18.443500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.443471    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"]
Oct 30 21:35:18.447050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.447030    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.447259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.447240    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.447305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.447273    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ds5f\" (UniqueName: \"kubernetes.io/projected/864a9c89-2cc3-4865-b8c9-49aa3a95564c-kube-api-access-5ds5f\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.548377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.548346    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.548498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.548459    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.548498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.548476    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ds5f\" (UniqueName: \"kubernetes.io/projected/864a9c89-2cc3-4865-b8c9-49aa3a95564c-kube-api-access-5ds5f\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.548943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.548919    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.550812 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.550791    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.605245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.605176    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ds5f\" (UniqueName: \"kubernetes.io/projected/864a9c89-2cc3-4865-b8c9-49aa3a95564c-kube-api-access-5ds5f\") pod \"maas-api-revoke-sub-843f6340-fwgzk\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:18.652335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:18.652309    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:19.033336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:19.033311    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"]
Oct 30 21:35:19.034192 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:19.034167    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864a9c89_2cc3_4865_b8c9_49aa3a95564c.slice/crio-a3e460befd188e84af4488566d286909f2cdd7abdf27ea3836b38912f117b604 WatchSource:0}: Error finding container a3e460befd188e84af4488566d286909f2cdd7abdf27ea3836b38912f117b604: Status 404 returned error can't find the container with id a3e460befd188e84af4488566d286909f2cdd7abdf27ea3836b38912f117b604
Oct 30 21:35:19.971052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:19.971025    5786 generic.go:356] "Generic (PLEG): container finished" podID="864a9c89-2cc3-4865-b8c9-49aa3a95564c" containerID="4f7b71f2c233a832c3843b9240a874bdb546bdab028d3d0e5436020d738780ec" exitCode=0
Oct 30 21:35:19.971387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:19.971106    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk" event={"ID":"864a9c89-2cc3-4865-b8c9-49aa3a95564c","Type":"ContainerDied","Data":"4f7b71f2c233a832c3843b9240a874bdb546bdab028d3d0e5436020d738780ec"}
Oct 30 21:35:19.971387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:19.971158    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk" event={"ID":"864a9c89-2cc3-4865-b8c9-49aa3a95564c","Type":"ContainerStarted","Data":"a3e460befd188e84af4488566d286909f2cdd7abdf27ea3836b38912f117b604"}
Oct 30 21:35:20.950844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:20.950814    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"]
Oct 30 21:35:20.955443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:20.955424    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.137232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.134757    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b25d536b-openshift-default-dockercfg-dxq4p\""
Oct 30 21:35:21.158125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.158087    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-credential-socket\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.158429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.158409    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j46qk\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-kube-api-access-j46qk\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.158628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.158612    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-podinfo\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.158756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.158741    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istiod-ca-cert\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.158902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.158884    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-certs\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.159122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.159108    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-socket\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.159270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.159254    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-data\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.159477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.159461    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-token\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.159583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.159565    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-envoy\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.195883 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.194670    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"]
Oct 30 21:35:21.307570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307444    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-podinfo\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.307570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307552    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istiod-ca-cert\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.307806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307626    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-certs\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.307806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307737    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-socket\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.307806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307764    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-data\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.307997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307823    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-token\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.307997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307849    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-envoy\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.307997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307924    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-credential-socket\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.307997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.307979    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j46qk\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-kube-api-access-j46qk\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.315021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.310940    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-data\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.315021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.311579    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istiod-ca-cert\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.315021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.311832    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-certs\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.315021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.312046    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-socket\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.331530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.322877    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-envoy\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.331530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.323427    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-credential-socket\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.331530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.325077    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-podinfo\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.384267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.383813    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-token\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.384890 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.384869    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j46qk\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-kube-api-access-j46qk\") pod \"e2e-ait-b25d536b-openshift-default-7974949447-tlf8p\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") " pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:21.471745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.471716    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:21.509575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.509553    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/864a9c89-2cc3-4865-b8c9-49aa3a95564c-kube-api-access-5ds5f\" (UniqueName: \"kubernetes.io/projected/864a9c89-2cc3-4865-b8c9-49aa3a95564c-kube-api-access-5ds5f\") pod \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") "
Oct 30 21:35:21.509751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.509735    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-xks-ca\") pod \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") "
Oct 30 21:35:21.509831 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.509815    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-openshift-ca\") pod \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\" (UID: \"864a9c89-2cc3-4865-b8c9-49aa3a95564c\") "
Oct 30 21:35:21.510121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.510100    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-openshift-ca" pod "864a9c89-2cc3-4865-b8c9-49aa3a95564c" (UID: "864a9c89-2cc3-4865-b8c9-49aa3a95564c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:21.510322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.510308    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:21.511605 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.511582    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/864a9c89-2cc3-4865-b8c9-49aa3a95564c-kube-api-access-5ds5f" pod "864a9c89-2cc3-4865-b8c9-49aa3a95564c" (UID: "864a9c89-2cc3-4865-b8c9-49aa3a95564c"). InnerVolumeSpecName "kube-api-access-5ds5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:21.511707 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.511686    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-xks-ca" pod "864a9c89-2cc3-4865-b8c9-49aa3a95564c" (UID: "864a9c89-2cc3-4865-b8c9-49aa3a95564c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:21.612444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.612374    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/864a9c89-2cc3-4865-b8c9-49aa3a95564c-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:21.612444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.612409    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ds5f\" (UniqueName: \"kubernetes.io/projected/864a9c89-2cc3-4865-b8c9-49aa3a95564c-kube-api-access-5ds5f\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:21.612582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:21.612532    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:22.016978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:22.016956    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"
Oct 30 21:35:22.017129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:22.016973    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk" event={"ID":"864a9c89-2cc3-4865-b8c9-49aa3a95564c","Type":"ContainerDied","Data":"a3e460befd188e84af4488566d286909f2cdd7abdf27ea3836b38912f117b604"}
Oct 30 21:35:22.017129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:22.017018    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3e460befd188e84af4488566d286909f2cdd7abdf27ea3836b38912f117b604"
Oct 30 21:35:22.046110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:22.046088    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"]
Oct 30 21:35:22.046377 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:22.046358    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411e28c0_bd7b_4294_b39f_0cb678be0fbf.slice/crio-f2be54dba858e55bf483227a9c30145493682c37dbd37f133ef96214751d0c7b WatchSource:0}: Error finding container f2be54dba858e55bf483227a9c30145493682c37dbd37f133ef96214751d0c7b: Status 404 returned error can't find the container with id f2be54dba858e55bf483227a9c30145493682c37dbd37f133ef96214751d0c7b
Oct 30 21:35:22.048705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:22.048671    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:22.048795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:22.048734    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:22.048795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:22.048761    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:22.355269 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:35:22.355234    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:35:23.027400 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:23.027367    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" event={"ID":"411e28c0-bd7b-4294-b39f-0cb678be0fbf","Type":"ContainerStarted","Data":"fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151"}
Oct 30 21:35:23.027551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:23.027420    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" event={"ID":"411e28c0-bd7b-4294-b39f-0cb678be0fbf","Type":"ContainerStarted","Data":"f2be54dba858e55bf483227a9c30145493682c37dbd37f133ef96214751d0c7b"}
Oct 30 21:35:23.195018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:23.194954    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podStartSLOduration=3.194935722 podStartE2EDuration="3.194935722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:23.192582969 +0000 UTC m=+1379.063702088" watchObservedRunningTime="2026-10-30 21:35:23.194935722 +0000 UTC m=+1379.066054841"
Oct 30 21:35:23.613085 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:23.613051    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:23.618603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:23.618582    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:24.035622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.035599    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:24.036407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.036388    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:35:24.634025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.633994    5786 scope.go:122] "RemoveContainer" containerID="91ee1d748940cd5344e2261615e0cb28a251c4d9f08814016a510e8d286dabbd"
Oct 30 21:35:24.645273 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.645258    5786 scope.go:122] "RemoveContainer" containerID="b6b7f7f58136d5f7bb83256ca014fe62ec9eaf799273929b1001d3188dc98204"
Oct 30 21:35:24.654920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.654904    5786 scope.go:122] "RemoveContainer" containerID="dc5210ab2b12fe007e0230b4d1e4029269934b0d43db031770349226d65c77ec"
Oct 30 21:35:24.663520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.663501    5786 scope.go:122] "RemoveContainer" containerID="d89a4b3d5968969a2807ecc3df72dbb289948222c39a93ac406b4f59c264ae8a"
Oct 30 21:35:24.673286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.673267    5786 scope.go:122] "RemoveContainer" containerID="2b49be44b9209e87d5d863f4fa706527a14a54f78bf79b2662ceed56c5a25cc8"
Oct 30 21:35:24.682704 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.682689    5786 scope.go:122] "RemoveContainer" containerID="6c9096be03f97b5a60cafa033ab70e9071aa1e34ad36d11bf3679ca179353113"
Oct 30 21:35:24.691263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.691248    5786 scope.go:122] "RemoveContainer" containerID="e8fee44c20d0d8e70ccdde8438b87614ca4b37b97911f0728b1f3487fd25621c"
Oct 30 21:35:24.699878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.699858    5786 scope.go:122] "RemoveContainer" containerID="4d333f0e9e387ffe9727fb7c01f4bee62c779719ee3524349ddece2837067929"
Oct 30 21:35:24.708431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:24.708416    5786 scope.go:122] "RemoveContainer" containerID="c7d8110844ac7a04e2ad671555f0a55d897c77aef5b301e2589da72e7fa91f88"
Oct 30 21:35:25.276912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.276882    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"]
Oct 30 21:35:25.277566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.277551    5786 cpu_manager.go:414] "containerMap: removing container" podUID="864a9c89-2cc3-4865-b8c9-49aa3a95564c" containerName="revoke-keys"
Oct 30 21:35:25.277633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.277567    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="864a9c89-2cc3-4865-b8c9-49aa3a95564c" containerName="revoke-keys"
Oct 30 21:35:25.277693 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.277683    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="864a9c89-2cc3-4865-b8c9-49aa3a95564c" containerName="revoke-keys"
Oct 30 21:35:25.283480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.283456    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.362200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.362167    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee854f35-35ea-4780-aaa6-2421b4026be3-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.362393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.362374    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.362439 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.362413    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.362484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.362459    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.362674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.362647    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.362873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.362845    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.362976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.362897    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56r2z\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-kube-api-access-56r2z\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.363083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.363064    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.363139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.363117    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.392618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.392590    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"]
Oct 30 21:35:25.463905 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.463877    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.463920    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.463969    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee854f35-35ea-4780-aaa6-2421b4026be3-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464015    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464039    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464069    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464113    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464184    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464275    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56r2z\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-kube-api-access-56r2z\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464430 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464347    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-certs\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464486 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464421    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-credential-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464610    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-data\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464630    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-socket\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.464985 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.464950    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee854f35-35ea-4780-aaa6-2421b4026be3-istiod-ca-cert\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.466602 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.466580    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-envoy\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.466680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.466639    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-podinfo\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.488416 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.488394    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"]
Oct 30 21:35:25.493845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.493826    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.520598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.520575    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-token\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.521390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.521370    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56r2z\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-kube-api-access-56r2z\") pod \"e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") " pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.553649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.553514    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"]
Oct 30 21:35:25.598159 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.598133    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:25.679897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.679864    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/688684ad-2f63-44f1-9855-636878fecb22-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.680353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.679912    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.680353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.679959    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.680353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.680076    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.680353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.680128    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.680353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.680155    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.680353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.680188    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/688684ad-2f63-44f1-9855-636878fecb22-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.680353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.680265    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.680353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.680320    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lx66\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-kube-api-access-4lx66\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781396    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781580 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781432    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781580 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781456    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/688684ad-2f63-44f1-9855-636878fecb22-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781580 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781475    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781580 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781519    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lx66\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-kube-api-access-4lx66\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781641    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/688684ad-2f63-44f1-9855-636878fecb22-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781767    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781819    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.781891 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781844    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.782026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.781999    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.782100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.782054    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.782157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.782136    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/688684ad-2f63-44f1-9855-636878fecb22-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.782242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.782165    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.782402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.782380    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.783667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.783648    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.784043 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.784026    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/688684ad-2f63-44f1-9855-636878fecb22-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.837793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.837745    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:25.837793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:25.837786    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lx66\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-kube-api-access-4lx66\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:26.082092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.082060    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"]
Oct 30 21:35:26.082960 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:26.082932    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee854f35_35ea_4780_aaa6_2421b4026be3.slice/crio-0f6aa6d36aa24c1fa447261725ea230d68b54ed6919700e678b2ed857e3197e1 WatchSource:0}: Error finding container 0f6aa6d36aa24c1fa447261725ea230d68b54ed6919700e678b2ed857e3197e1: Status 404 returned error can't find the container with id 0f6aa6d36aa24c1fa447261725ea230d68b54ed6919700e678b2ed857e3197e1
Oct 30 21:35:26.085465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.085430    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:26.085545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.085502    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:26.085578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.085540    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:26.108223 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.108183    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:26.492376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.492351    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"]
Oct 30 21:35:26.495966 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:26.495938    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod688684ad_2f63_44f1_9855_636878fecb22.slice/crio-5d992c0327920bc6d7440d440993e1af08ebec64e39baaa28fa555cc454aeab6 WatchSource:0}: Error finding container 5d992c0327920bc6d7440d440993e1af08ebec64e39baaa28fa555cc454aeab6: Status 404 returned error can't find the container with id 5d992c0327920bc6d7440d440993e1af08ebec64e39baaa28fa555cc454aeab6
Oct 30 21:35:26.498897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.498858    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:26.498981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.498946    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:26.499021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:26.498988    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:27.079231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.076921    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" event={"ID":"688684ad-2f63-44f1-9855-636878fecb22","Type":"ContainerStarted","Data":"b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215"}
Oct 30 21:35:27.079231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.077225    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" event={"ID":"688684ad-2f63-44f1-9855-636878fecb22","Type":"ContainerStarted","Data":"5d992c0327920bc6d7440d440993e1af08ebec64e39baaa28fa555cc454aeab6"}
Oct 30 21:35:27.084260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.083083    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" event={"ID":"ee854f35-35ea-4780-aaa6-2421b4026be3","Type":"ContainerStarted","Data":"7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201"}
Oct 30 21:35:27.084260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.083139    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" event={"ID":"ee854f35-35ea-4780-aaa6-2421b4026be3","Type":"ContainerStarted","Data":"0f6aa6d36aa24c1fa447261725ea230d68b54ed6919700e678b2ed857e3197e1"}
Oct 30 21:35:27.109194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.108527    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:27.111744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.111317    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:35:27.111744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.111384    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 21:35:27.209233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.200483    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podStartSLOduration=2.200464181 podStartE2EDuration="2.200464181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:27.198598798 +0000 UTC m=+1383.069717917" watchObservedRunningTime="2026-10-30 21:35:27.200464181 +0000 UTC m=+1383.071583301"
Oct 30 21:35:27.319876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.319826    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podStartSLOduration=2.319810957 podStartE2EDuration="2.319810957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:27.314896769 +0000 UTC m=+1383.186015887" watchObservedRunningTime="2026-10-30 21:35:27.319810957 +0000 UTC m=+1383.190930075"
Oct 30 21:35:27.602230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.601561    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:27.606225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.605464    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:35:27.606225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:27.605519    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 21:35:28.119256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:28.111611    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:35:28.119256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:28.111681    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 21:35:28.605232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:28.600561    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:35:28.605232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:28.600625    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 21:35:29.113262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.112498    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:35:29.113262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.112549    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 21:35:29.305590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.305550    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"]
Oct 30 21:35:29.310108 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.310090    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.322103 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.322078    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.322233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.322122    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.322233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.322145    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhjds\" (UniqueName: \"kubernetes.io/projected/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-kube-api-access-fhjds\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.326534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.326517    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"]
Oct 30 21:35:29.331502 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.331252    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.348109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.348086    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"]
Oct 30 21:35:29.386817 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.386734    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1d5266a5-openshift-default-dockercfg-kbjt2\""
Oct 30 21:35:29.447271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.388069    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:35:29.832308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.400395    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:35:29.832615 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.674306    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:35:29.832802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.832778    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 21:35:29.832967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.686146    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-token\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.833042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833022    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-data\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.833371 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833327    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-socket\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.833554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833536    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.833643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833595    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-podinfo\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.833700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833662    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.833758 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833717    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjds\" (UniqueName: \"kubernetes.io/projected/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-kube-api-access-fhjds\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.833842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833827    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-envoy\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.833896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833874    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-credential-socket\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.834011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.833997    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istiod-ca-cert\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.834272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.834250    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-certs\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.834360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.834348    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvjzh\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-kube-api-access-hvjzh\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.835003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.834979    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.840285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.838892    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.845128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.845084    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"]
Oct 30 21:35:29.902376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.902334    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhjds\" (UniqueName: \"kubernetes.io/projected/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-kube-api-access-fhjds\") pod \"maas-api-revoke-sub-a0c8aad3-pr5zr\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.924985 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.924963    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:29.937372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937020    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istiod-ca-cert\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.937372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937114    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-certs\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.937372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937142    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvjzh\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-kube-api-access-hvjzh\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.937372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937262    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-token\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.937372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937290    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-data\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.937372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937324    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-socket\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.937372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937374    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-podinfo\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.937900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937533    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-envoy\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.937900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.937565    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-credential-socket\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.939512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.939487    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istiod-ca-cert\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.940113 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.939921    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-credential-socket\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.940256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.940229    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-certs\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.940415 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.940397    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-data\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.940510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.940408    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-socket\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.943549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.943507    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-podinfo\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:29.944245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:29.944160    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-envoy\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:30.078227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:30.072251    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvjzh\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-kube-api-access-hvjzh\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:30.078227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:30.073651    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-token\") pod \"e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") " pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:30.125394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:30.124867    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:35:30.125394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:30.124922    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 21:35:30.170952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:30.170923    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:30.628234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:30.626249    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:35:30.628234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:30.626316    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 21:35:30.844561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:30.844531    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"]
Oct 30 21:35:30.845107 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:30.845081    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc849a498_3ab9_41c2_bc85_5c8ca6ef22e9.slice/crio-8f3a1c4a9d006fa76e82cbaf03d8cf33e0ee47fcd167f8f6046a9007b2992cbe WatchSource:0}: Error finding container 8f3a1c4a9d006fa76e82cbaf03d8cf33e0ee47fcd167f8f6046a9007b2992cbe: Status 404 returned error can't find the container with id 8f3a1c4a9d006fa76e82cbaf03d8cf33e0ee47fcd167f8f6046a9007b2992cbe
Oct 30 21:35:31.041589 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:31.041559    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf64b51_cf8b_47ca_81e6_d9365f1de030.slice/crio-8a706cf0dcd9b152cb5228c0f262ac5b7d7c05766f744c23b1ead4d7990f3671 WatchSource:0}: Error finding container 8a706cf0dcd9b152cb5228c0f262ac5b7d7c05766f744c23b1ead4d7990f3671: Status 404 returned error can't find the container with id 8a706cf0dcd9b152cb5228c0f262ac5b7d7c05766f744c23b1ead4d7990f3671
Oct 30 21:35:31.043849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.043821    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"]
Oct 30 21:35:31.044529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.044495    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:31.044592 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.044576    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:31.044637 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.044623    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:31.109697 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.109670    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:35:31.109955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.109921    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 21:35:31.223180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.223148    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr" event={"ID":"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9","Type":"ContainerStarted","Data":"8f3a1c4a9d006fa76e82cbaf03d8cf33e0ee47fcd167f8f6046a9007b2992cbe"}
Oct 30 21:35:31.224671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.224650    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" event={"ID":"dcf64b51-cf8b-47ca-81e6-d9365f1de030","Type":"ContainerStarted","Data":"8a706cf0dcd9b152cb5228c0f262ac5b7d7c05766f744c23b1ead4d7990f3671"}
Oct 30 21:35:31.606233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.606067    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:35:31.606233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:31.606126    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 21:35:32.118753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:32.118712    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:35:32.119377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:32.118787    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 21:35:32.234872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:32.234844    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" event={"ID":"dcf64b51-cf8b-47ca-81e6-d9365f1de030","Type":"ContainerStarted","Data":"f9c905dac5f0dd5dd12dd21645e9b0d14acf80b7e528b2bfab47291ad8beec26"}
Oct 30 21:35:32.237112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:32.237090    5786 generic.go:356] "Generic (PLEG): container finished" podID="c849a498-3ab9-41c2-bc85-5c8ca6ef22e9" containerID="2ab3d25ed983556b10e22ad9af981704fa29c1f368f46d1a7655df97f01c5ea7" exitCode=0
Oct 30 21:35:32.237287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:32.237275    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr" event={"ID":"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9","Type":"ContainerDied","Data":"2ab3d25ed983556b10e22ad9af981704fa29c1f368f46d1a7655df97f01c5ea7"}
Oct 30 21:35:32.409368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:32.409272    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podStartSLOduration=3.409254432 podStartE2EDuration="3.409254432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:32.360647444 +0000 UTC m=+1388.231766565" watchObservedRunningTime="2026-10-30 21:35:32.409254432 +0000 UTC m=+1388.280373542"
Oct 30 21:35:32.603291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:32.599068    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:35:32.603291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:32.599126    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 21:35:33.108966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.108932    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:35:33.109128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.108980    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 21:35:33.180790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.180751    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:33.186102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.186075    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:33.245830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.245796    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:33.246989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.246961    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:35:33.428355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.428325    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:33.494407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.494372    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-openshift-ca\") pod \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") "
Oct 30 21:35:33.494407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.494444    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-xks-ca\") pod \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") "
Oct 30 21:35:33.494407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.494774    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-kube-api-access-fhjds\" (UniqueName: \"kubernetes.io/projected/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-kube-api-access-fhjds\") pod \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\" (UID: \"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9\") "
Oct 30 21:35:33.495509 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.494803    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-openshift-ca" pod "c849a498-3ab9-41c2-bc85-5c8ca6ef22e9" (UID: "c849a498-3ab9-41c2-bc85-5c8ca6ef22e9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:33.497766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.497700    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-xks-ca" pod "c849a498-3ab9-41c2-bc85-5c8ca6ef22e9" (UID: "c849a498-3ab9-41c2-bc85-5c8ca6ef22e9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:33.497766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.497714    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-kube-api-access-fhjds" pod "c849a498-3ab9-41c2-bc85-5c8ca6ef22e9" (UID: "c849a498-3ab9-41c2-bc85-5c8ca6ef22e9"). InnerVolumeSpecName "kube-api-access-fhjds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:33.597133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.597043    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhjds\" (UniqueName: \"kubernetes.io/projected/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-kube-api-access-fhjds\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:33.597133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.597084    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:33.597133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.597102    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:33.599586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.599558    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:35:33.599717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:33.599615    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 21:35:34.109102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.109071    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:35:34.109306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.109123    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 21:35:34.187185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.187152    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"]
Oct 30 21:35:34.188256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.188234    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c849a498-3ab9-41c2-bc85-5c8ca6ef22e9" containerName="revoke-keys"
Oct 30 21:35:34.188310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.188260    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c849a498-3ab9-41c2-bc85-5c8ca6ef22e9" containerName="revoke-keys"
Oct 30 21:35:34.188504 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.188489    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c849a498-3ab9-41c2-bc85-5c8ca6ef22e9" containerName="revoke-keys"
Oct 30 21:35:34.193467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.193444    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.269898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.269866    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"]
Oct 30 21:35:34.272426 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.272405    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"
Oct 30 21:35:34.272590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.272426    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr" event={"ID":"c849a498-3ab9-41c2-bc85-5c8ca6ef22e9","Type":"ContainerDied","Data":"8f3a1c4a9d006fa76e82cbaf03d8cf33e0ee47fcd167f8f6046a9007b2992cbe"}
Oct 30 21:35:34.272709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.272697    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f3a1c4a9d006fa76e82cbaf03d8cf33e0ee47fcd167f8f6046a9007b2992cbe"
Oct 30 21:35:34.313554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.313428    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.313554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.313496    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.313554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.313551    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57qpc\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-kube-api-access-57qpc\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.313880 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.313753    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.313880 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.313804    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6f2e61a-2162-4698-acea-b14639ddff20-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.314004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.313928    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.314004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.313979    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.314115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.314030    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.314177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.314153    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6f2e61a-2162-4698-acea-b14639ddff20-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415316    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415353    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415382    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57qpc\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-kube-api-access-57qpc\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415527    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415563    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6f2e61a-2162-4698-acea-b14639ddff20-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415624    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415668    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415752    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415762    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.415998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.415877    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6f2e61a-2162-4698-acea-b14639ddff20-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.416251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.416226    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.416403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.416351    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.416557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.416536    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.416557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.416492    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6f2e61a-2162-4698-acea-b14639ddff20-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.418165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.418142    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.418680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.418662    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6f2e61a-2162-4698-acea-b14639ddff20-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.473502 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.473477    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.473673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.473652    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57qpc\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-kube-api-access-57qpc\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.509407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.509379    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:34.610435 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.610410    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:34.611625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.611605    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:34.613457 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.613434    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:35:34.981551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.981516    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"]
Oct 30 21:35:34.986166 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:34.986134    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f2e61a_2162_4698_acea_b14639ddff20.slice/crio-5902f55d19c3f206a61933f5a33966ec2529a41978bd8f82f671c48fde6a676f WatchSource:0}: Error finding container 5902f55d19c3f206a61933f5a33966ec2529a41978bd8f82f671c48fde6a676f: Status 404 returned error can't find the container with id 5902f55d19c3f206a61933f5a33966ec2529a41978bd8f82f671c48fde6a676f
Oct 30 21:35:34.988874 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.988840    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:34.988966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.988911    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:34.988966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:34.988943    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:35.114231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.114190    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:35.114756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.114729    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:35.116069 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.116049    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:35:35.282539 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.282460    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" event={"ID":"b6f2e61a-2162-4698-acea-b14639ddff20","Type":"ContainerStarted","Data":"70eba89230707eb27224e7e2e1c724702716f761af70a2a927b236a4c395440b"}
Oct 30 21:35:35.282539 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.282511    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" event={"ID":"b6f2e61a-2162-4698-acea-b14639ddff20","Type":"ContainerStarted","Data":"5902f55d19c3f206a61933f5a33966ec2529a41978bd8f82f671c48fde6a676f"}
Oct 30 21:35:35.471249 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.471177    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podStartSLOduration=1.471162919 podStartE2EDuration="1.471162919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:35.467406377 +0000 UTC m=+1391.338525497" watchObservedRunningTime="2026-10-30 21:35:35.471162919 +0000 UTC m=+1391.342282037"
Oct 30 21:35:35.510548 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.510513    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:35.907229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.903864    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"]
Oct 30 21:35:35.912243 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.909865    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:35.976232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.969541    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:35:35.976232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.970271    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:35:35.982796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:35.982757    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"]
Oct 30 21:35:36.048754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.048716    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvf9g\" (UniqueName: \"kubernetes.io/projected/484f95a6-d51d-43f6-903d-8879bbb0cd2e-kube-api-access-lvf9g\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.048983 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.048964    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.049059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.049001    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.150805 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.150765    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.150805 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.150810    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.151043 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.150953    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvf9g\" (UniqueName: \"kubernetes.io/projected/484f95a6-d51d-43f6-903d-8879bbb0cd2e-kube-api-access-lvf9g\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.157276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.151536    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.176978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.176942    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.237280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.220258    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvf9g\" (UniqueName: \"kubernetes.io/projected/484f95a6-d51d-43f6-903d-8879bbb0cd2e-kube-api-access-lvf9g\") pod \"maas-api-revoke-sub-cc3dc3ab-fwcj7\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.237280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.225695    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:36.511763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.511730    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:35:36.512216 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.511830    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:35:36.680075 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:36.680047    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"]
Oct 30 21:35:37.302169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.302096    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7" event={"ID":"484f95a6-d51d-43f6-903d-8879bbb0cd2e","Type":"ContainerStarted","Data":"bacd070c02c02b0d4e9caa1abdd9ee006c061e7ac7773e64caa8ca512c367866"}
Oct 30 21:35:37.302169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.302149    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7" event={"ID":"484f95a6-d51d-43f6-903d-8879bbb0cd2e","Type":"ContainerStarted","Data":"c2ced40f894027af6f0862fb94fc27084a87701b8a47621845c3a4ed3db8f079"}
Oct 30 21:35:37.440585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.440536    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7" podStartSLOduration=2.440522278 podStartE2EDuration="2.440522278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:37.436812684 +0000 UTC m=+1393.307931803" watchObservedRunningTime="2026-10-30 21:35:37.440522278 +0000 UTC m=+1393.311641396"
Oct 30 21:35:37.510456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.510424    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:35:37.510608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.510473    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:35:37.616019 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.615937    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"]
Oct 30 21:35:37.621018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.620996    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.651094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.651061    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.675238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.674643    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pd7c\" (UniqueName: \"kubernetes.io/projected/0785e604-6eaf-4e7c-be09-90130c919abb-kube-api-access-7pd7c\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.675238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.674707    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.702410 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.702376    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"]
Oct 30 21:35:37.777586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.776095    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.777586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.776977    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.777586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.777167    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pd7c\" (UniqueName: \"kubernetes.io/projected/0785e604-6eaf-4e7c-be09-90130c919abb-kube-api-access-7pd7c\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.778288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.778258    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.781687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.781665    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.835122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.835093    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pd7c\" (UniqueName: \"kubernetes.io/projected/0785e604-6eaf-4e7c-be09-90130c919abb-kube-api-access-7pd7c\") pod \"maas-api-revoke-sub-8b170718-zj9tf\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:37.935324 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:37.935221    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:38.381338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:38.380633    5786 generic.go:356] "Generic (PLEG): container finished" podID="484f95a6-d51d-43f6-903d-8879bbb0cd2e" containerID="bacd070c02c02b0d4e9caa1abdd9ee006c061e7ac7773e64caa8ca512c367866" exitCode=0
Oct 30 21:35:38.487230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:38.464990    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7" event={"ID":"484f95a6-d51d-43f6-903d-8879bbb0cd2e","Type":"ContainerDied","Data":"bacd070c02c02b0d4e9caa1abdd9ee006c061e7ac7773e64caa8ca512c367866"}
Oct 30 21:35:38.541756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:38.540339    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:35:38.541756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:38.540409    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:35:38.570391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:38.569785    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"]
Oct 30 21:35:39.014387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.014359    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"]
Oct 30 21:35:39.030372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.030347    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"]
Oct 30 21:35:39.030561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.030541    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:39.082522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.082487    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1d5266a5\""
Oct 30 21:35:39.082670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.082539    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1d5266a5-dockercfg-ngzzt\""
Oct 30 21:35:39.119014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.118985    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5fpv\" (UniqueName: \"kubernetes.io/projected/a192c0be-4988-4876-a080-a890a9f360e8-kube-api-access-f5fpv\") pod \"payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf\" (UID: \"a192c0be-4988-4876-a080-a890a9f360e8\") " pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:39.119125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.119029    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a192c0be-4988-4876-a080-a890a9f360e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf\" (UID: \"a192c0be-4988-4876-a080-a890a9f360e8\") " pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:39.220132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.220060    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5fpv\" (UniqueName: \"kubernetes.io/projected/a192c0be-4988-4876-a080-a890a9f360e8-kube-api-access-f5fpv\") pod \"payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf\" (UID: \"a192c0be-4988-4876-a080-a890a9f360e8\") " pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:39.220132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.220113    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a192c0be-4988-4876-a080-a890a9f360e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf\" (UID: \"a192c0be-4988-4876-a080-a890a9f360e8\") " pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:39.221008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.220982    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a192c0be-4988-4876-a080-a890a9f360e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf\" (UID: \"a192c0be-4988-4876-a080-a890a9f360e8\") " pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:39.281666 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.281637    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5fpv\" (UniqueName: \"kubernetes.io/projected/a192c0be-4988-4876-a080-a890a9f360e8-kube-api-access-f5fpv\") pod \"payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf\" (UID: \"a192c0be-4988-4876-a080-a890a9f360e8\") " pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:39.383704 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.383670    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:39.515604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.515493    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:35:39.515604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.515555    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:35:39.523252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.522769    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf" event={"ID":"0785e604-6eaf-4e7c-be09-90130c919abb","Type":"ContainerStarted","Data":"300d16c9db49257492e709e7687bb5df0136757bea43a59cfb415fccba13340a"}
Oct 30 21:35:39.523252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.522824    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf" event={"ID":"0785e604-6eaf-4e7c-be09-90130c919abb","Type":"ContainerStarted","Data":"3dda7f6a30b0d77fc959084915436efbb0fdbbc7fd6b7eb3bc9a5b632cf52a65"}
Oct 30 21:35:39.609496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.609455    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"]
Oct 30 21:35:39.618067 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.617593    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.636771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.636744    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.636894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.636820    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.636894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.636854    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.637008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.636938    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.637008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.636972    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040f7639-1a65-4901-bda0-8f7c852045a3-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.637008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.637006    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhxdg\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-kube-api-access-lhxdg\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.637147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.637026    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.637147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.637043    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.637147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.637107    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040f7639-1a65-4901-bda0-8f7c852045a3-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.645301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.645250    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf" podStartSLOduration=2.645233508 podStartE2EDuration="2.645233508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:39.642709434 +0000 UTC m=+1395.513828580" watchObservedRunningTime="2026-10-30 21:35:39.645233508 +0000 UTC m=+1395.516352629"
Oct 30 21:35:39.708341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.708308    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"]
Oct 30 21:35:39.738962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.738936    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040f7639-1a65-4901-bda0-8f7c852045a3-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.739272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.739250    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.739594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.739573    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.739868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.739849    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.739951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.739850    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040f7639-1a65-4901-bda0-8f7c852045a3-istiod-ca-cert\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.739951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.739935    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-data\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.740058 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.739981    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.740110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.740083    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.740162 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.740137    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040f7639-1a65-4901-bda0-8f7c852045a3-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.740307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.740189    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhxdg\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-kube-api-access-lhxdg\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.740631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.740234    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.740708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.740658    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.741364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.741342    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-credential-socket\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.741537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.741519    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-certs\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.748735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.748713    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040f7639-1a65-4901-bda0-8f7c852045a3-istio-podinfo\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.749030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.749008    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-envoy\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.782331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.780641    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:39.797819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.797792    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhxdg\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-kube-api-access-lhxdg\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.798328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.798303    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-istio-token\") pod \"e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") " pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.842527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.842505    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/484f95a6-d51d-43f6-903d-8879bbb0cd2e-kube-api-access-lvf9g\" (UniqueName: \"kubernetes.io/projected/484f95a6-d51d-43f6-903d-8879bbb0cd2e-kube-api-access-lvf9g\") pod \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") "
Oct 30 21:35:39.842638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.842551    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-openshift-ca\") pod \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") "
Oct 30 21:35:39.842638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.842588    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-xks-ca\") pod \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\" (UID: \"484f95a6-d51d-43f6-903d-8879bbb0cd2e\") "
Oct 30 21:35:39.842900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.842873    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-openshift-ca" pod "484f95a6-d51d-43f6-903d-8879bbb0cd2e" (UID: "484f95a6-d51d-43f6-903d-8879bbb0cd2e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:39.843155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.843140    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:39.844603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.844582    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/484f95a6-d51d-43f6-903d-8879bbb0cd2e-kube-api-access-lvf9g" pod "484f95a6-d51d-43f6-903d-8879bbb0cd2e" (UID: "484f95a6-d51d-43f6-903d-8879bbb0cd2e"). InnerVolumeSpecName "kube-api-access-lvf9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:39.844668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.844622    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-xks-ca" pod "484f95a6-d51d-43f6-903d-8879bbb0cd2e" (UID: "484f95a6-d51d-43f6-903d-8879bbb0cd2e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:39.927963 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.927939    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"]
Oct 30 21:35:39.929808 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:39.929779    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda192c0be_4988_4876_a080_a890a9f360e8.slice/crio-9bff5d1b068f8c69c4a58a4595c979e2faa2889212f5c58ad2fcafccd246a9c8 WatchSource:0}: Error finding container 9bff5d1b068f8c69c4a58a4595c979e2faa2889212f5c58ad2fcafccd246a9c8: Status 404 returned error can't find the container with id 9bff5d1b068f8c69c4a58a4595c979e2faa2889212f5c58ad2fcafccd246a9c8
Oct 30 21:35:39.944101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.944071    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:39.988685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.988224    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvf9g\" (UniqueName: \"kubernetes.io/projected/484f95a6-d51d-43f6-903d-8879bbb0cd2e-kube-api-access-lvf9g\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:39.991183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:39.991160    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/484f95a6-d51d-43f6-903d-8879bbb0cd2e-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:40.481187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.481154    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"]
Oct 30 21:35:40.482752 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:40.482728    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040f7639_1a65_4901_bda0_8f7c852045a3.slice/crio-67b71698e57c2184cd625692f69ea398498416a90f410ea0ed1274a4fc84a1ad WatchSource:0}: Error finding container 67b71698e57c2184cd625692f69ea398498416a90f410ea0ed1274a4fc84a1ad: Status 404 returned error can't find the container with id 67b71698e57c2184cd625692f69ea398498416a90f410ea0ed1274a4fc84a1ad
Oct 30 21:35:40.485866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.485835    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:40.485966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.485898    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:40.485966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.485929    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:40.509830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.509808    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:35:40.509933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.509850    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:35:40.533719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.533694    5786 generic.go:356] "Generic (PLEG): container finished" podID="0785e604-6eaf-4e7c-be09-90130c919abb" containerID="300d16c9db49257492e709e7687bb5df0136757bea43a59cfb415fccba13340a" exitCode=0
Oct 30 21:35:40.533809 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.533775    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf" event={"ID":"0785e604-6eaf-4e7c-be09-90130c919abb","Type":"ContainerDied","Data":"300d16c9db49257492e709e7687bb5df0136757bea43a59cfb415fccba13340a"}
Oct 30 21:35:40.535877 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.535851    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf" event={"ID":"a192c0be-4988-4876-a080-a890a9f360e8","Type":"ContainerStarted","Data":"6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b"}
Oct 30 21:35:40.535985 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.535904    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf" event={"ID":"a192c0be-4988-4876-a080-a890a9f360e8","Type":"ContainerStarted","Data":"9bff5d1b068f8c69c4a58a4595c979e2faa2889212f5c58ad2fcafccd246a9c8"}
Oct 30 21:35:40.536046 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.535994    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:40.537365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.537347    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" event={"ID":"040f7639-1a65-4901-bda0-8f7c852045a3","Type":"ContainerStarted","Data":"67b71698e57c2184cd625692f69ea398498416a90f410ea0ed1274a4fc84a1ad"}
Oct 30 21:35:40.538732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.538714    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7" event={"ID":"484f95a6-d51d-43f6-903d-8879bbb0cd2e","Type":"ContainerDied","Data":"c2ced40f894027af6f0862fb94fc27084a87701b8a47621845c3a4ed3db8f079"}
Oct 30 21:35:40.538832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.538750    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2ced40f894027af6f0862fb94fc27084a87701b8a47621845c3a4ed3db8f079"
Oct 30 21:35:40.538832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:40.538761    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"
Oct 30 21:35:41.509983 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.509945    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:35:41.510375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.509996    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:35:41.548508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.548473    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" event={"ID":"040f7639-1a65-4901-bda0-8f7c852045a3","Type":"ContainerStarted","Data":"b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8"}
Oct 30 21:35:41.699349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.696336    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf" podStartSLOduration=3.696310032 podStartE2EDuration="3.696310032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:40.806507114 +0000 UTC m=+1396.677626247" watchObservedRunningTime="2026-10-30 21:35:41.696310032 +0000 UTC m=+1397.567429151"
Oct 30 21:35:41.699349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.698554    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podStartSLOduration=2.698538531 podStartE2EDuration="2.698538531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:41.672085457 +0000 UTC m=+1397.543204579" watchObservedRunningTime="2026-10-30 21:35:41.698538531 +0000 UTC m=+1397.569657652"
Oct 30 21:35:41.744544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.744511    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:41.878972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.878882    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0785e604-6eaf-4e7c-be09-90130c919abb-kube-api-access-7pd7c\" (UniqueName: \"kubernetes.io/projected/0785e604-6eaf-4e7c-be09-90130c919abb-kube-api-access-7pd7c\") pod \"0785e604-6eaf-4e7c-be09-90130c919abb\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") "
Oct 30 21:35:41.879308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.879284    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-xks-ca\") pod \"0785e604-6eaf-4e7c-be09-90130c919abb\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") "
Oct 30 21:35:41.879618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.879602    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-openshift-ca\") pod \"0785e604-6eaf-4e7c-be09-90130c919abb\" (UID: \"0785e604-6eaf-4e7c-be09-90130c919abb\") "
Oct 30 21:35:41.888430 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.880746    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-openshift-ca" pod "0785e604-6eaf-4e7c-be09-90130c919abb" (UID: "0785e604-6eaf-4e7c-be09-90130c919abb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:41.891653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.891604    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-xks-ca" pod "0785e604-6eaf-4e7c-be09-90130c919abb" (UID: "0785e604-6eaf-4e7c-be09-90130c919abb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:41.894087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.894052    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0785e604-6eaf-4e7c-be09-90130c919abb-kube-api-access-7pd7c" pod "0785e604-6eaf-4e7c-be09-90130c919abb" (UID: "0785e604-6eaf-4e7c-be09-90130c919abb"). InnerVolumeSpecName "kube-api-access-7pd7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:41.946604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.946279    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:41.947537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.947510    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:41.947628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.947560    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:41.982569 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.982523    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:41.982569 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.982567    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pd7c\" (UniqueName: \"kubernetes.io/projected/0785e604-6eaf-4e7c-be09-90130c919abb-kube-api-access-7pd7c\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:41.982772 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:41.982583    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0785e604-6eaf-4e7c-be09-90130c919abb-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:42.582377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:42.579603    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"
Oct 30 21:35:42.583676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:42.582268    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf" event={"ID":"0785e604-6eaf-4e7c-be09-90130c919abb","Type":"ContainerDied","Data":"3dda7f6a30b0d77fc959084915436efbb0fdbbc7fd6b7eb3bc9a5b632cf52a65"}
Oct 30 21:35:42.583676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:42.583106    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3dda7f6a30b0d77fc959084915436efbb0fdbbc7fd6b7eb3bc9a5b632cf52a65"
Oct 30 21:35:42.622011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:42.621982    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:42.622474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:42.622449    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:42.623455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:42.623437    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:35:42.946118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:42.946084    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:42.946404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:42.946137    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:43.977235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:43.973315    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:43.977235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:43.973380    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:44.250285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.249274    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"]
Oct 30 21:35:44.250467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.250307    5786 cpu_manager.go:414] "containerMap: removing container" podUID="484f95a6-d51d-43f6-903d-8879bbb0cd2e" containerName="revoke-keys"
Oct 30 21:35:44.250467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.250322    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="484f95a6-d51d-43f6-903d-8879bbb0cd2e" containerName="revoke-keys"
Oct 30 21:35:44.250467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.250338    5786 cpu_manager.go:414] "containerMap: removing container" podUID="0785e604-6eaf-4e7c-be09-90130c919abb" containerName="revoke-keys"
Oct 30 21:35:44.250467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.250345    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0785e604-6eaf-4e7c-be09-90130c919abb" containerName="revoke-keys"
Oct 30 21:35:44.250691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.250529    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="484f95a6-d51d-43f6-903d-8879bbb0cd2e" containerName="revoke-keys"
Oct 30 21:35:44.250691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.250545    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="0785e604-6eaf-4e7c-be09-90130c919abb" containerName="revoke-keys"
Oct 30 21:35:44.277041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.259296    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"]
Oct 30 21:35:44.277041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.259406    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:44.313577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.313528    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-6f787654\""
Oct 30 21:35:44.313762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.313744    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-6f787654-dockercfg-8j26m\""
Oct 30 21:35:44.398995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.398958    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d1c57f4-80c3-4c3c-b679-de2945760f76-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc\" (UID: \"8d1c57f4-80c3-4c3c-b679-de2945760f76\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:44.399178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.399078    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2rr9\" (UniqueName: \"kubernetes.io/projected/8d1c57f4-80c3-4c3c-b679-de2945760f76-kube-api-access-j2rr9\") pod \"payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc\" (UID: \"8d1c57f4-80c3-4c3c-b679-de2945760f76\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:44.500135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.500108    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2rr9\" (UniqueName: \"kubernetes.io/projected/8d1c57f4-80c3-4c3c-b679-de2945760f76-kube-api-access-j2rr9\") pod \"payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc\" (UID: \"8d1c57f4-80c3-4c3c-b679-de2945760f76\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:44.500430 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.500377    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d1c57f4-80c3-4c3c-b679-de2945760f76-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc\" (UID: \"8d1c57f4-80c3-4c3c-b679-de2945760f76\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:44.501044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.501010    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d1c57f4-80c3-4c3c-b679-de2945760f76-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc\" (UID: \"8d1c57f4-80c3-4c3c-b679-de2945760f76\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:44.580021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.579991    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2rr9\" (UniqueName: \"kubernetes.io/projected/8d1c57f4-80c3-4c3c-b679-de2945760f76-kube-api-access-j2rr9\") pod \"payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc\" (UID: \"8d1c57f4-80c3-4c3c-b679-de2945760f76\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:44.613082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.612836    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:44.947354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.947324    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:44.947584 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:44.947554    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:45.036042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:45.036018    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"]
Oct 30 21:35:45.036844 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:45.036821    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d1c57f4_80c3_4c3c_b679_de2945760f76.slice/crio-de157b633aed310580e7898ab70593e44409d6d0b98751be963b130ebe401729 WatchSource:0}: Error finding container de157b633aed310580e7898ab70593e44409d6d0b98751be963b130ebe401729: Status 404 returned error can't find the container with id de157b633aed310580e7898ab70593e44409d6d0b98751be963b130ebe401729
Oct 30 21:35:45.716266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:45.716199    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc" event={"ID":"8d1c57f4-80c3-4c3c-b679-de2945760f76","Type":"ContainerStarted","Data":"71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034"}
Oct 30 21:35:45.716449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:45.716287    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc" event={"ID":"8d1c57f4-80c3-4c3c-b679-de2945760f76","Type":"ContainerStarted","Data":"de157b633aed310580e7898ab70593e44409d6d0b98751be963b130ebe401729"}
Oct 30 21:35:45.716449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:45.716430    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:45.837034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:45.836981    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc" podStartSLOduration=2.836966906 podStartE2EDuration="2.836966906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:45.835471709 +0000 UTC m=+1401.706590830" watchObservedRunningTime="2026-10-30 21:35:45.836966906 +0000 UTC m=+1401.708086023"
Oct 30 21:35:45.946471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:45.946434    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:45.946912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:45.946858    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:46.946360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:46.946323    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:46.946790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:46.946387    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:47.946086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:47.946047    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:47.946312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:47.946092    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:48.386189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.386096    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"]
Oct 30 21:35:48.392311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.392286    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.447384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.447355    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"]
Oct 30 21:35:48.471468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.471427    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:35:48.488681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.488658    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:35:48.520885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.520859    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtj7c\" (UniqueName: \"kubernetes.io/projected/ea826efb-93df-4ca2-8a61-f2036c25d462-kube-api-access-rtj7c\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.521028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.520984    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.521076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.521032    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.622490 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.622463    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtj7c\" (UniqueName: \"kubernetes.io/projected/ea826efb-93df-4ca2-8a61-f2036c25d462-kube-api-access-rtj7c\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.622644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.622523    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.622644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.622553    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.623137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.623118    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.624924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.624902    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.678851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.678826    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtj7c\" (UniqueName: \"kubernetes.io/projected/ea826efb-93df-4ca2-8a61-f2036c25d462-kube-api-access-rtj7c\") pod \"maas-api-revoke-sub-a88fe2c3-v7cg4\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.706484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.706465    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:48.862739 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.862710    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"]
Oct 30 21:35:48.868184 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.868165    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.942979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.941489    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7379f500-openshift-default-dockercfg-lj7s7\""
Oct 30 21:35:48.946715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.946681    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65c2976-ed99-4599-93cd-c1fa38b5df34-istiod-ca-cert\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.946873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.946801    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-socket\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.946934 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.946884    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-certs\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.946994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.946969    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-podinfo\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.947046 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.947027    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54khr\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-kube-api-access-54khr\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.947096 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.947081    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-credential-socket\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.949660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.947192    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-token\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.949660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.947352    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-data\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.949660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.947427    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-envoy\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:48.998232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.989525    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:48.998232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.990885    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:48.998232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:48.997427    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.049702    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65c2976-ed99-4599-93cd-c1fa38b5df34-istiod-ca-cert\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.049842    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-socket\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.050058    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-certs\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.050174    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-podinfo\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.050306    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54khr\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-kube-api-access-54khr\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.050375    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-credential-socket\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.050534    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-token\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.050574    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-data\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.050632    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-envoy\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.051122    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"]
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.051495    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-certs\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.052119    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65c2976-ed99-4599-93cd-c1fa38b5df34-istiod-ca-cert\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.052600    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-socket\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.057516    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-podinfo\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.058168    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-credential-socket\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.059593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.058422    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-data\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.109842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.082674    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-envoy\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.277014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.276983    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54khr\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-kube-api-access-54khr\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.296691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.286644    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-token\") pod \"e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") " pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.356452 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:49.356329    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea826efb_93df_4ca2_8a61_f2036c25d462.slice/crio-921d722eb87b865d4c30f78d64e1e3b8651a707f8576dbf49af9cded599c57d8 WatchSource:0}: Error finding container 921d722eb87b865d4c30f78d64e1e3b8651a707f8576dbf49af9cded599c57d8: Status 404 returned error can't find the container with id 921d722eb87b865d4c30f78d64e1e3b8651a707f8576dbf49af9cded599c57d8
Oct 30 21:35:49.388516 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.387397    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"]
Oct 30 21:35:49.576280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.570327    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:49.836524 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:49.836436    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4" event={"ID":"ea826efb-93df-4ca2-8a61-f2036c25d462","Type":"ContainerStarted","Data":"921d722eb87b865d4c30f78d64e1e3b8651a707f8576dbf49af9cded599c57d8"}
Oct 30 21:35:50.821232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:50.814989    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:50.958494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:50.957015    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:50.960298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:50.959963    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:50.984302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:50.981533    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:50.984302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:50.981629    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:50.984302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:50.981677    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:35:50.991118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:50.989115    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"]
Oct 30 21:35:51.019468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.019433    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" event={"ID":"c65c2976-ed99-4599-93cd-c1fa38b5df34","Type":"ContainerStarted","Data":"557cf1a1833436969d4df03914903f741bc33a00f2997b39f4bde507a7f034e8"}
Oct 30 21:35:51.023413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.023378    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4" event={"ID":"ea826efb-93df-4ca2-8a61-f2036c25d462","Type":"ContainerStarted","Data":"14860fe0754bdba9bc9942a04906060a49a301b6e0898356000645085970d810"}
Oct 30 21:35:51.041397 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.040833    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:51.183282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.161376    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4" podStartSLOduration=3.161359297 podStartE2EDuration="3.161359297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:51.157990858 +0000 UTC m=+1407.029109979" watchObservedRunningTime="2026-10-30 21:35:51.161359297 +0000 UTC m=+1407.032478418"
Oct 30 21:35:51.255505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.255279    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:51.568599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.567370    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:35:51.612885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.611826    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:51.789289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.694408    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/health\": context deadline exceeded"
Oct 30 21:35:51.789289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.708290    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:51.801343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.800562    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/health\": context deadline exceeded"
Oct 30 21:35:51.932161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.931773    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:51.941713 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.941310    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:51.969251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.961311    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:52.070316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.970088    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:35:52.070316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.971333    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:35:52.070316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:51.987242    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:35:52.099308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.077368    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:35:52.099308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.077684    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:35:52.167460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.167415    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:52.251472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.236946    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/health\": context deadline exceeded"
Oct 30 21:35:52.252805 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.252755    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:52.256184 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.253000    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:52.283908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.281294    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:52.283908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.282535    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:35:52.618410 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.618367    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:52.620198 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.620094    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:35:52.762411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.737314    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": context deadline exceeded"
Oct 30 21:35:52.814267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.792303    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:35:52.839232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.832314    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:35:52.933398 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.933358    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": context deadline exceeded"
Oct 30 21:35:52.935392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.935368    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:35:52.945309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.945277    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 30 21:35:52.951812 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.951787    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:35:52.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.967962    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:52.982498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:52.974318    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:35:53.044235 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:35:53.025277    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:35:53.127287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.081959    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:53.515376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.271597    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:35:53.515376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.272102    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:35:53.515376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.277499    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:35:53.515376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.288485    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:35:53.515376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.292685    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:53.515376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.457550    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:53.515376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.492307    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:35:53.515376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.492366    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:53.620233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.619494    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:53.816330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.816284    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": context deadline exceeded"
Oct 30 21:35:53.955447 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:53.955405    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 30 21:35:54.072338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.011387    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:35:54.072338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.011863    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": context deadline exceeded"
Oct 30 21:35:54.072338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.012366    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:35:54.120278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.120239    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:35:54.121036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.120986    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:54.354290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.310494    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:35:54.402235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.376859    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:35:54.552235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.425761    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:35:54.865336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.714303    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" event={"ID":"c65c2976-ed99-4599-93cd-c1fa38b5df34","Type":"ContainerStarted","Data":"c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f"}
Oct 30 21:35:54.865336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.768549    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:54.865336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.792505    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:35:54.966365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:54.966309    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:55.102991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.102940    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": context deadline exceeded"
Oct 30 21:35:55.107302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.103269    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:35:55.129462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.129359    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:35:55.129790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.129763    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 21:35:55.360273 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.313518    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:55.514233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.513310    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:55.532267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.532194    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:35:55.532570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.532556    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:35:55.553673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.553641    5786 generic.go:356] "Generic (PLEG): container finished" podID="ea826efb-93df-4ca2-8a61-f2036c25d462" containerID="14860fe0754bdba9bc9942a04906060a49a301b6e0898356000645085970d810" exitCode=0
Oct 30 21:35:55.557585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.557414    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4" event={"ID":"ea826efb-93df-4ca2-8a61-f2036c25d462","Type":"ContainerDied","Data":"14860fe0754bdba9bc9942a04906060a49a301b6e0898356000645085970d810"}
Oct 30 21:35:55.576121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.576084    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:55.592895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:55.592860    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:56.378354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.378299    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 21:35:56.417233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.400464    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:56.417755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.417714    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:56.430231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.418507    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:35:56.430231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.418828    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": context deadline exceeded"
Oct 30 21:35:56.430231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.419074    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:56.430231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.419331    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:35:56.430231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.419572    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:35:56.430231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.426712    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:35:56.430231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.426973    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:35:56.430231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.427196    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/health\": context deadline exceeded"
Oct 30 21:35:56.639313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.531685    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:56.741285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.711388    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:56.951934 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.951902    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:35:57.031536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:56.959617    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:57.453643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.397354    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:57.455949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.413375    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:57.456644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.438777    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:57.603301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.457785    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:57.603301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.459005    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:35:57.611404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.461270    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 21:35:57.611558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.611409    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:35:57.621398 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.469448    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": context deadline exceeded"
Oct 30 21:35:57.631932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.631904    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:35:57.648234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.469595    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": context deadline exceeded"
Oct 30 21:35:57.649702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.469621    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:35:57.666248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.506904    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:57.666248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.558225    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podStartSLOduration=9.558183757 podStartE2EDuration="9.558183757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:56.568449889 +0000 UTC m=+1412.439569010" watchObservedRunningTime="2026-10-30 21:35:57.558183757 +0000 UTC m=+1413.429302875"
Oct 30 21:35:57.672704 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.469403    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:57.737713 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.737601    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:57.785586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.785554    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:35:57.811233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.809009    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:35:57.851235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:57.840256    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:35:58.110282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.109304    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:35:58.110282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.109395    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:35:58.177291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.176931    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:35:58.181324 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.178513    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:35:58.193335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.193315    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"]
Oct 30 21:35:58.212916 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.212890    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"]
Oct 30 21:35:58.213136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.213120    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:35:58.213312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.213291    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:35:58.213440 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.213428    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:35:58.213548 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.213538    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:35:58.213691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.213680    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:35:58.214643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.214625    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.232982 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.232951    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h782h\" (UniqueName: \"kubernetes.io/projected/95ff2755-4034-4522-b09d-6b46fe8f5a63-kube-api-access-h782h\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.234355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.234326    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.235337 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.235317    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.290814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.290792    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:58.336516 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.336488    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea826efb-93df-4ca2-8a61-f2036c25d462-kube-api-access-rtj7c\" (UniqueName: \"kubernetes.io/projected/ea826efb-93df-4ca2-8a61-f2036c25d462-kube-api-access-rtj7c\") pod \"ea826efb-93df-4ca2-8a61-f2036c25d462\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") "
Oct 30 21:35:58.336690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.336596    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-openshift-ca\") pod \"ea826efb-93df-4ca2-8a61-f2036c25d462\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") "
Oct 30 21:35:58.336741 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.336718    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-xks-ca\") pod \"ea826efb-93df-4ca2-8a61-f2036c25d462\" (UID: \"ea826efb-93df-4ca2-8a61-f2036c25d462\") "
Oct 30 21:35:58.337035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.337005    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-openshift-ca" pod "ea826efb-93df-4ca2-8a61-f2036c25d462" (UID: "ea826efb-93df-4ca2-8a61-f2036c25d462"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:58.337419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.337238    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.337419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.337389    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h782h\" (UniqueName: \"kubernetes.io/projected/95ff2755-4034-4522-b09d-6b46fe8f5a63-kube-api-access-h782h\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.337770 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.337736    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.337874 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.337841    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:58.338853 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.338826    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.340236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.340196    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea826efb-93df-4ca2-8a61-f2036c25d462-kube-api-access-rtj7c" pod "ea826efb-93df-4ca2-8a61-f2036c25d462" (UID: "ea826efb-93df-4ca2-8a61-f2036c25d462"). InnerVolumeSpecName "kube-api-access-rtj7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:58.340463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.340397    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-xks-ca" pod "ea826efb-93df-4ca2-8a61-f2036c25d462" (UID: "ea826efb-93df-4ca2-8a61-f2036c25d462"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:58.340669 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.340648    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.394301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.394277    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h782h\" (UniqueName: \"kubernetes.io/projected/95ff2755-4034-4522-b09d-6b46fe8f5a63-kube-api-access-h782h\") pod \"maas-api-revoke-sub-aac4ddbb-htdpm\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.440084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.440051    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtj7c\" (UniqueName: \"kubernetes.io/projected/ea826efb-93df-4ca2-8a61-f2036c25d462-kube-api-access-rtj7c\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:58.440193 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.440090    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea826efb-93df-4ca2-8a61-f2036c25d462-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:58.563173 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.553094    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:35:58.952967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:58.952936    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"]
Oct 30 21:35:58.953257 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:35:58.953194    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ff2755_4034_4522_b09d_6b46fe8f5a63.slice/crio-d12ab2731fbd6f6345c504e47648600c7819e701967574b0c3e1d791f722e9b5 WatchSource:0}: Error finding container d12ab2731fbd6f6345c504e47648600c7819e701967574b0c3e1d791f722e9b5: Status 404 returned error can't find the container with id d12ab2731fbd6f6345c504e47648600c7819e701967574b0c3e1d791f722e9b5
Oct 30 21:35:59.215848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:59.215747    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4" event={"ID":"ea826efb-93df-4ca2-8a61-f2036c25d462","Type":"ContainerDied","Data":"921d722eb87b865d4c30f78d64e1e3b8651a707f8576dbf49af9cded599c57d8"}
Oct 30 21:35:59.215848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:59.215820    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="921d722eb87b865d4c30f78d64e1e3b8651a707f8576dbf49af9cded599c57d8"
Oct 30 21:35:59.215848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:59.215772    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"
Oct 30 21:35:59.217172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:35:59.217147    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm" event={"ID":"95ff2755-4034-4522-b09d-6b46fe8f5a63","Type":"ContainerStarted","Data":"d12ab2731fbd6f6345c504e47648600c7819e701967574b0c3e1d791f722e9b5"}
Oct 30 21:36:00.226883 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:00.226852    5786 generic.go:356] "Generic (PLEG): container finished" podID="95ff2755-4034-4522-b09d-6b46fe8f5a63" containerID="48d20a1556bb1ec385bc32d7c03c64e330901738546fdb54714b19dfa9f51c66" exitCode=0
Oct 30 21:36:00.227300 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:00.226928    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm" event={"ID":"95ff2755-4034-4522-b09d-6b46fe8f5a63","Type":"ContainerDied","Data":"48d20a1556bb1ec385bc32d7c03c64e330901738546fdb54714b19dfa9f51c66"}
Oct 30 21:36:01.402485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.402456    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:36:01.489180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.489120    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-openshift-ca\") pod \"95ff2755-4034-4522-b09d-6b46fe8f5a63\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") "
Oct 30 21:36:01.489326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.489200    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95ff2755-4034-4522-b09d-6b46fe8f5a63-kube-api-access-h782h\" (UniqueName: \"kubernetes.io/projected/95ff2755-4034-4522-b09d-6b46fe8f5a63-kube-api-access-h782h\") pod \"95ff2755-4034-4522-b09d-6b46fe8f5a63\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") "
Oct 30 21:36:01.489411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.489396    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-xks-ca\") pod \"95ff2755-4034-4522-b09d-6b46fe8f5a63\" (UID: \"95ff2755-4034-4522-b09d-6b46fe8f5a63\") "
Oct 30 21:36:01.489493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.489456    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-openshift-ca" pod "95ff2755-4034-4522-b09d-6b46fe8f5a63" (UID: "95ff2755-4034-4522-b09d-6b46fe8f5a63"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:01.489926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.489908    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:36:01.491364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.491341    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95ff2755-4034-4522-b09d-6b46fe8f5a63-kube-api-access-h782h" pod "95ff2755-4034-4522-b09d-6b46fe8f5a63" (UID: "95ff2755-4034-4522-b09d-6b46fe8f5a63"). InnerVolumeSpecName "kube-api-access-h782h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:01.491437 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.491410    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-xks-ca" pod "95ff2755-4034-4522-b09d-6b46fe8f5a63" (UID: "95ff2755-4034-4522-b09d-6b46fe8f5a63"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:36:01.591250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.591189    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h782h\" (UniqueName: \"kubernetes.io/projected/95ff2755-4034-4522-b09d-6b46fe8f5a63-kube-api-access-h782h\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:36:01.591250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:01.591249    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95ff2755-4034-4522-b09d-6b46fe8f5a63-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:36:02.286644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.286586    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm" event={"ID":"95ff2755-4034-4522-b09d-6b46fe8f5a63","Type":"ContainerDied","Data":"d12ab2731fbd6f6345c504e47648600c7819e701967574b0c3e1d791f722e9b5"}
Oct 30 21:36:02.286886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.286868    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d12ab2731fbd6f6345c504e47648600c7819e701967574b0c3e1d791f722e9b5"
Oct 30 21:36:02.287034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.287023    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"
Oct 30 21:36:02.522286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.522254    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"]
Oct 30 21:36:02.537573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.537484    5786 cpu_manager.go:414] "containerMap: removing container" podUID="95ff2755-4034-4522-b09d-6b46fe8f5a63" containerName="revoke-keys"
Oct 30 21:36:02.555309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.555277    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95ff2755-4034-4522-b09d-6b46fe8f5a63" containerName="revoke-keys"
Oct 30 21:36:02.621227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.620829    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ea826efb-93df-4ca2-8a61-f2036c25d462" containerName="revoke-keys"
Oct 30 21:36:02.621227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.621029    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea826efb-93df-4ca2-8a61-f2036c25d462" containerName="revoke-keys"
Oct 30 21:36:02.641229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.621932    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea826efb-93df-4ca2-8a61-f2036c25d462" containerName="revoke-keys"
Oct 30 21:36:02.641229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.621967    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="95ff2755-4034-4522-b09d-6b46fe8f5a63" containerName="revoke-keys"
Oct 30 21:36:02.735549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.735515    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"]
Oct 30 21:36:02.735895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.735870    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:02.788519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.745977    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17677e40-4d28-49c1-8138-bd25a18e23f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5\" (UID: \"17677e40-4d28-49c1-8138-bd25a18e23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:02.788519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.746051    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqqnj\" (UniqueName: \"kubernetes.io/projected/17677e40-4d28-49c1-8138-bd25a18e23f5-kube-api-access-zqqnj\") pod \"payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5\" (UID: \"17677e40-4d28-49c1-8138-bd25a18e23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:02.788519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.779644    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"]
Oct 30 21:36:02.788519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.786069    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:02.789428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.788946    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7379f500-dockercfg-7kn9g\""
Oct 30 21:36:02.792310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.792283    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7379f500\""
Oct 30 21:36:02.802480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.802158    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"]
Oct 30 21:36:02.851484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.851449    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17677e40-4d28-49c1-8138-bd25a18e23f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5\" (UID: \"17677e40-4d28-49c1-8138-bd25a18e23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:02.851599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.851518    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc54c61-f740-45fc-81c8-a5c2412c06e7-plugins-config-volume\") pod \"payload-processing-e2e-ait-7379f500-644568b6d9-k45x9\" (UID: \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\") " pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:02.851599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.851548    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqqnj\" (UniqueName: \"kubernetes.io/projected/17677e40-4d28-49c1-8138-bd25a18e23f5-kube-api-access-zqqnj\") pod \"payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5\" (UID: \"17677e40-4d28-49c1-8138-bd25a18e23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:02.851785 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.851770    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4mkq\" (UniqueName: \"kubernetes.io/projected/8bc54c61-f740-45fc-81c8-a5c2412c06e7-kube-api-access-p4mkq\") pod \"payload-processing-e2e-ait-7379f500-644568b6d9-k45x9\" (UID: \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\") " pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:02.852279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.852258    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17677e40-4d28-49c1-8138-bd25a18e23f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5\" (UID: \"17677e40-4d28-49c1-8138-bd25a18e23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:02.909033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.909002    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqqnj\" (UniqueName: \"kubernetes.io/projected/17677e40-4d28-49c1-8138-bd25a18e23f5-kube-api-access-zqqnj\") pod \"payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5\" (UID: \"17677e40-4d28-49c1-8138-bd25a18e23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:02.952518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.952483    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc54c61-f740-45fc-81c8-a5c2412c06e7-plugins-config-volume\") pod \"payload-processing-e2e-ait-7379f500-644568b6d9-k45x9\" (UID: \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\") " pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:02.952692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.952675    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4mkq\" (UniqueName: \"kubernetes.io/projected/8bc54c61-f740-45fc-81c8-a5c2412c06e7-kube-api-access-p4mkq\") pod \"payload-processing-e2e-ait-7379f500-644568b6d9-k45x9\" (UID: \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\") " pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:02.953150 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:02.953130    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc54c61-f740-45fc-81c8-a5c2412c06e7-plugins-config-volume\") pod \"payload-processing-e2e-ait-7379f500-644568b6d9-k45x9\" (UID: \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\") " pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:03.009482 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:03.009462    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4mkq\" (UniqueName: \"kubernetes.io/projected/8bc54c61-f740-45fc-81c8-a5c2412c06e7-kube-api-access-p4mkq\") pod \"payload-processing-e2e-ait-7379f500-644568b6d9-k45x9\" (UID: \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\") " pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:03.117427 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:03.117359    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:03.128690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:03.128656    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:03.505602 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:36:03.505574    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17677e40_4d28_49c1_8138_bd25a18e23f5.slice/crio-499c551378b06463f81ee1e9e72c1439367d02c4f21551463af5eb0bdbbc21ed WatchSource:0}: Error finding container 499c551378b06463f81ee1e9e72c1439367d02c4f21551463af5eb0bdbbc21ed: Status 404 returned error can't find the container with id 499c551378b06463f81ee1e9e72c1439367d02c4f21551463af5eb0bdbbc21ed
Oct 30 21:36:03.506445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:03.506423    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"]
Oct 30 21:36:03.555047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:03.555020    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"]
Oct 30 21:36:03.555403 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:36:03.555132    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc54c61_f740_45fc_81c8_a5c2412c06e7.slice/crio-9aebb7fc03d30f646a65406231ef36757cd7809979c6552fec87f8f5d4b7d743 WatchSource:0}: Error finding container 9aebb7fc03d30f646a65406231ef36757cd7809979c6552fec87f8f5d4b7d743: Status 404 returned error can't find the container with id 9aebb7fc03d30f646a65406231ef36757cd7809979c6552fec87f8f5d4b7d743
Oct 30 21:36:04.309277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:04.309240    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9" event={"ID":"8bc54c61-f740-45fc-81c8-a5c2412c06e7","Type":"ContainerStarted","Data":"c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806"}
Oct 30 21:36:04.309450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:04.309297    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9" event={"ID":"8bc54c61-f740-45fc-81c8-a5c2412c06e7","Type":"ContainerStarted","Data":"9aebb7fc03d30f646a65406231ef36757cd7809979c6552fec87f8f5d4b7d743"}
Oct 30 21:36:04.309450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:04.309367    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:04.311118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:04.311093    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5" event={"ID":"17677e40-4d28-49c1-8138-bd25a18e23f5","Type":"ContainerStarted","Data":"7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666"}
Oct 30 21:36:04.311266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:04.311136    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5" event={"ID":"17677e40-4d28-49c1-8138-bd25a18e23f5","Type":"ContainerStarted","Data":"499c551378b06463f81ee1e9e72c1439367d02c4f21551463af5eb0bdbbc21ed"}
Oct 30 21:36:04.311266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:04.311174    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:04.424199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:04.424146    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9" podStartSLOduration=2.424133104 podStartE2EDuration="2.424133104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:04.421541506 +0000 UTC m=+1420.292660629" watchObservedRunningTime="2026-10-30 21:36:04.424133104 +0000 UTC m=+1420.295252223"
Oct 30 21:36:04.590298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:04.583066    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5" podStartSLOduration=2.5830453589999998 podStartE2EDuration="2.583045359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:04.577441083 +0000 UTC m=+1420.448560264" watchObservedRunningTime="2026-10-30 21:36:04.583045359 +0000 UTC m=+1420.454164480"
Oct 30 21:36:10.899529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.899501    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"]
Oct 30 21:36:10.903915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.903899    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.966982    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.967085    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr72s\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-kube-api-access-zr72s\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967239 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.967142    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.967257    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.967307    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.967363    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.967415    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.967505    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.967624 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.967605    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:10.998995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:10.998967    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"]
Oct 30 21:36:11.069233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069172    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069283    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr72s\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-kube-api-access-zr72s\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069349    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069381    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069427    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069462    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069520    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069590    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069729 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069720    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069794    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-data\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069839    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-certs\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.069978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.069958    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.070038 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.070007    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-credential-socket\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.070292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.070272    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istiod-ca-cert\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.072018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.071992    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-envoy\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.072252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.072228    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-podinfo\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.125261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.125232    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-token\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.125377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.125286    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr72s\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-kube-api-access-zr72s\") pod \"e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") " pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.218528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.218506    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:11.665531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.665501    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"]
Oct 30 21:36:11.690045 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:36:11.690020    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15a6df7_ab5b_4b57_8a78_0e54a806e305.slice/crio-16d7961cc814f1ca6bfac4407e4857cc3aa7b37f5b78edc9e96270a5cce3290a WatchSource:0}: Error finding container 16d7961cc814f1ca6bfac4407e4857cc3aa7b37f5b78edc9e96270a5cce3290a: Status 404 returned error can't find the container with id 16d7961cc814f1ca6bfac4407e4857cc3aa7b37f5b78edc9e96270a5cce3290a
Oct 30 21:36:11.692508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.692477    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:11.692590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.692540    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:11.692590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:11.692570    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:12.064431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.064358    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"]
Oct 30 21:36:12.069030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.069011    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.122225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.121799    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b25d536b-serving-cert\""
Oct 30 21:36:12.130432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.130405    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"]
Oct 30 21:36:12.185723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.185686    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.186080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.186056    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwckj\" (UniqueName: \"kubernetes.io/projected/1b56cbbb-7ddd-46ea-a284-ff469f5240af-kube-api-access-zwckj\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.186199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.186126    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-metrics-certs\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.186199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.186163    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b56cbbb-7ddd-46ea-a284-ff469f5240af-openshift-service-ca\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.287353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.287311    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwckj\" (UniqueName: \"kubernetes.io/projected/1b56cbbb-7ddd-46ea-a284-ff469f5240af-kube-api-access-zwckj\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.287562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.287391    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-metrics-certs\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.287562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.287430    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b56cbbb-7ddd-46ea-a284-ff469f5240af-openshift-service-ca\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.287562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.287494    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.287720 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:12.287647    5786 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-serving-cert: secret "maas-api-e2e-ait-b25d536b-serving-cert" not found
Oct 30 21:36:12.287777 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:12.287731    5786 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls podName:1b56cbbb-7ddd-46ea-a284-ff469f5240af nodeName:}" failed. No retries permitted until 2026-10-30 21:36:12.787708035 +0000 UTC m=+1428.658827145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls") pod "maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f" (UID: "1b56cbbb-7ddd-46ea-a284-ff469f5240af") : secret "maas-api-e2e-ait-b25d536b-serving-cert" not found
Oct 30 21:36:12.288173 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.288148    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b56cbbb-7ddd-46ea-a284-ff469f5240af-openshift-service-ca\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.289907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.289888    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-metrics-certs\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.362036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.361939    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwckj\" (UniqueName: \"kubernetes.io/projected/1b56cbbb-7ddd-46ea-a284-ff469f5240af-kube-api-access-zwckj\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.536498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.536454    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" event={"ID":"b15a6df7-ab5b-4b57-8a78-0e54a806e305","Type":"ContainerStarted","Data":"5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079"}
Oct 30 21:36:12.536782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.536746    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" event={"ID":"b15a6df7-ab5b-4b57-8a78-0e54a806e305","Type":"ContainerStarted","Data":"16d7961cc814f1ca6bfac4407e4857cc3aa7b37f5b78edc9e96270a5cce3290a"}
Oct 30 21:36:12.688233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.681436    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" podStartSLOduration=2.681416549 podStartE2EDuration="2.681416549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:12.677978306 +0000 UTC m=+1428.549097425" watchObservedRunningTime="2026-10-30 21:36:12.681416549 +0000 UTC m=+1428.552535668"
Oct 30 21:36:12.809664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.809626    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.812163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.812141    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls\") pod \"maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:12.998963 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:12.998877    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:13.219380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:13.219340    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:13.220492 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:13.220468    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:36:13.220632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:13.220520    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:36:13.399823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:13.399795    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"]
Oct 30 21:36:13.399938 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:36:13.399860    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b56cbbb_7ddd_46ea_a284_ff469f5240af.slice/crio-9f9106bfac90753f04cf9a2e9c443ff9b75d95408ca404493e0da52719fe4db2 WatchSource:0}: Error finding container 9f9106bfac90753f04cf9a2e9c443ff9b75d95408ca404493e0da52719fe4db2: Status 404 returned error can't find the container with id 9f9106bfac90753f04cf9a2e9c443ff9b75d95408ca404493e0da52719fe4db2
Oct 30 21:36:13.401432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:13.401416    5786 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:36:13.545599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:13.545559    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f" event={"ID":"1b56cbbb-7ddd-46ea-a284-ff469f5240af","Type":"ContainerStarted","Data":"9f9106bfac90753f04cf9a2e9c443ff9b75d95408ca404493e0da52719fe4db2"}
Oct 30 21:36:14.232254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:14.229724    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:36:14.232254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:14.229791    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:36:14.579265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:14.579152    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f" event={"ID":"1b56cbbb-7ddd-46ea-a284-ff469f5240af","Type":"ContainerStarted","Data":"0aa91506720d587defcd4d7bd0174442d6a6bf63e114ca36b5f8bd4eb9959732"}
Oct 30 21:36:14.579265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:14.579262    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:14.695998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:14.695950    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f" podStartSLOduration=1.797179432 podStartE2EDuration="2.69593631s" totalImagesPullingTime="898.756878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:14.693788945 +0000 UTC m=+1430.564908078" watchObservedRunningTime="2026-10-30 21:36:14.69593631 +0000 UTC m=+1430.567055420"
Oct 30 21:36:15.230232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:15.221673    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:36:15.230232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:15.221753    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:36:15.382797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:15.377384    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:36:15.482546 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:15.482458    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:36:16.231700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:16.231664    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:36:16.231993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:16.231958    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:36:17.219722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:17.219686    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:36:17.220091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:17.219744    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:36:18.219836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:18.219801    5786 patch_prober.go:29] interesting pod/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:36:18.220252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:18.219864    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:36:19.223005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.222977    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:19.223462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.223440    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:19.224174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.224155    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:36:19.558534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.558450    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"]
Oct 30 21:36:19.582624 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.582591    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.648932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.648894    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.649118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.649002    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.649118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.649050    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.649118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.649069    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18da9832-9fe6-4381-96a2-539560c6663b-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.649405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.649185    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18da9832-9fe6-4381-96a2-539560c6663b-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.649405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.649325    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.649405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.649376    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.649562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.649492    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfxtz\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-kube-api-access-xfxtz\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.649562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.649511    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.661771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.661742    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"]
Oct 30 21:36:19.750643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.750611    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfxtz\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-kube-api-access-xfxtz\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.750810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.750648    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.750810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.750763    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.750928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.750855    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.750928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.750917    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.750947    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18da9832-9fe6-4381-96a2-539560c6663b-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.751010    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18da9832-9fe6-4381-96a2-539560c6663b-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.751065    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.751115    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.751134    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.751138    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.751602    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751904 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.751771    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18da9832-9fe6-4381-96a2-539560c6663b-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.751980 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.751965    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.753380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.753355    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.753636 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.753618    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18da9832-9fe6-4381-96a2-539560c6663b-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.806765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.806736    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.806888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.806820    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfxtz\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-kube-api-access-xfxtz\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:19.899023 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:19.898971    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:20.329890 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:36:20.329857    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18da9832_9fe6_4381_96a2_539560c6663b.slice/crio-981deafda8c7fe66ce8507e4dc5ff11ad14805c048e0bb426fc8b5725f08a284 WatchSource:0}: Error finding container 981deafda8c7fe66ce8507e4dc5ff11ad14805c048e0bb426fc8b5725f08a284: Status 404 returned error can't find the container with id 981deafda8c7fe66ce8507e4dc5ff11ad14805c048e0bb426fc8b5725f08a284
Oct 30 21:36:20.330288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.330073    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"]
Oct 30 21:36:20.332365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.332336    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:20.332455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.332393    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:20.332455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.332422    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:20.600676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.600604    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:36:20.648978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.648948    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" event={"ID":"18da9832-9fe6-4381-96a2-539560c6663b","Type":"ContainerStarted","Data":"7ecf255d8f1f506c4ef53848330dc1276635e16e083e2128b4ee51322afd2299"}
Oct 30 21:36:20.649131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.649004    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" event={"ID":"18da9832-9fe6-4381-96a2-539560c6663b","Type":"ContainerStarted","Data":"981deafda8c7fe66ce8507e4dc5ff11ad14805c048e0bb426fc8b5725f08a284"}
Oct 30 21:36:20.840856 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.840766    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" podStartSLOduration=1.8407455609999999 podStartE2EDuration="1.840745561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:20.8362346 +0000 UTC m=+1436.707353723" watchObservedRunningTime="2026-10-30 21:36:20.840745561 +0000 UTC m=+1436.711864682"
Oct 30 21:36:20.899664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:20.899589    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:21.711896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.711867    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"]
Oct 30 21:36:21.716643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.716622    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.783759 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.783724    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h4zs\" (UniqueName: \"kubernetes.io/projected/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-kube-api-access-5h4zs\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.784141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.784119    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.784330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.784313    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-openshift-service-ca\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.784517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.784502    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-metrics-certs\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.785747 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.785719    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"]
Oct 30 21:36:21.826894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.826865    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1d5266a5-serving-cert\""
Oct 30 21:36:21.886512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.886478    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.886706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.886561    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-openshift-service-ca\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.886706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.886669    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-metrics-certs\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.886882 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.886844    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h4zs\" (UniqueName: \"kubernetes.io/projected/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-kube-api-access-5h4zs\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.887618 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:21.887585    5786 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-serving-cert: secret "maas-api-e2e-ait-1d5266a5-serving-cert" not found
Oct 30 21:36:21.887852 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:21.887839    5786 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls podName:c7ae7dd9-4560-4a29-8425-0acc03ff57d5 nodeName:}" failed. No retries permitted until 2026-10-30 21:36:22.387800438 +0000 UTC m=+1438.258919557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls") pod "maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm" (UID: "c7ae7dd9-4560-4a29-8425-0acc03ff57d5") : secret "maas-api-e2e-ait-1d5266a5-serving-cert" not found
Oct 30 21:36:21.888006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.887982    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-openshift-service-ca\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.890697 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.890673    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-metrics-certs\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:21.900302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.900279    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:36:21.900419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.900332    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:36:21.949542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:21.949516    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h4zs\" (UniqueName: \"kubernetes.io/projected/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-kube-api-access-5h4zs\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:22.384967 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:22.384929    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:36:22.435715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:22.435690    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:22.438163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:22.438140    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls\") pod \"maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:22.631467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:22.631440    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:22.900736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:22.900646    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:36:22.900736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:22.900721    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:36:23.042607 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:36:23.042579    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ae7dd9_4560_4a29_8425_0acc03ff57d5.slice/crio-e853248f10afc173bf5e5bd7218f01d30ca4675ab1e47fe5f0e7e99d2166d00c WatchSource:0}: Error finding container e853248f10afc173bf5e5bd7218f01d30ca4675ab1e47fe5f0e7e99d2166d00c: Status 404 returned error can't find the container with id e853248f10afc173bf5e5bd7218f01d30ca4675ab1e47fe5f0e7e99d2166d00c
Oct 30 21:36:23.044706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:23.044680    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"]
Oct 30 21:36:23.698994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:23.698966    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm" event={"ID":"c7ae7dd9-4560-4a29-8425-0acc03ff57d5","Type":"ContainerStarted","Data":"e853248f10afc173bf5e5bd7218f01d30ca4675ab1e47fe5f0e7e99d2166d00c"}
Oct 30 21:36:23.907596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:23.906345    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:36:23.907596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:23.906406    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:36:24.709109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:24.709074    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm" event={"ID":"c7ae7dd9-4560-4a29-8425-0acc03ff57d5","Type":"ContainerStarted","Data":"a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306"}
Oct 30 21:36:24.709307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:24.709160    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:24.794230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:24.786660    5786 scope.go:122] "RemoveContainer" containerID="7bba7e233d430188d0dc95641d0ca2019da29593717af2c1316982eb1d7737ef"
Oct 30 21:36:24.855233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:24.824438    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"]
Oct 30 21:36:24.908117 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:24.908085    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"]
Oct 30 21:36:24.919418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:24.919389    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.009862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.009391    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:36:25.009862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.009447    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:36:25.011514 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.011491    5786 scope.go:122] "RemoveContainer" containerID="d14157cb15f622060ddd0be9e9a118fed78982690d54b5a74deec9d3a22cf9ea"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029019    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0910c2f7-ba5e-48fe-baaa-087201160f94-istiod-ca-cert\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029099    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-certs\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029220    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-token\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029254    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-credential-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029302    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029371    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5rb\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-kube-api-access-sq5rb\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029420    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-data\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029481    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-envoy\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.035231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.029563    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-podinfo\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150192    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0910c2f7-ba5e-48fe-baaa-087201160f94-istiod-ca-cert\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150291    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-certs\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150398    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-token\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150427    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-credential-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150484    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150554    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5rb\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-kube-api-access-sq5rb\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150602    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-data\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150663    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-envoy\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.152750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.150749    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-podinfo\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.211254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.207901    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0910c2f7-ba5e-48fe-baaa-087201160f94-istiod-ca-cert\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.211254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.208550    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-certs\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.211254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.209716    5786 scope.go:122] "RemoveContainer" containerID="8fac0ce7cd1a6366420530298703304af58af7b7129b7ccd3e6bf89707e861b0"
Oct 30 21:36:25.211254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.209912    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-credential-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.215015 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.214985    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-podinfo\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.216959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.216917    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-data\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.219893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.219724    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm" podStartSLOduration=3.271013504 podStartE2EDuration="4.219710515s" totalImagesPullingTime="948.697011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:24.891139236 +0000 UTC m=+1440.762258356" watchObservedRunningTime="2026-10-30 21:36:25.219710515 +0000 UTC m=+1441.090829635"
Oct 30 21:36:25.220794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.220760    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-envoy\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.238817 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.238792    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-socket\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.247068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.247052    5786 scope.go:122] "RemoveContainer" containerID="5fe3d1a18d828035d3754a0dfd9f31e3827224e56f5f0c6c986db52c1a154a29"
Oct 30 21:36:25.280385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.280326    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5rb\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-kube-api-access-sq5rb\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.281412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.281390    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-token\") pod \"e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") " pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.536504 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.536436    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:25.911295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.906379    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:36:25.911295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.906451    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:36:25.999109 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:36:25.999076    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0910c2f7_ba5e_48fe_baaa_087201160f94.slice/crio-2ec17d0e698cd1662a3e20accfcc25d5f9796d0411043009080ce6e34841c090 WatchSource:0}: Error finding container 2ec17d0e698cd1662a3e20accfcc25d5f9796d0411043009080ce6e34841c090: Status 404 returned error can't find the container with id 2ec17d0e698cd1662a3e20accfcc25d5f9796d0411043009080ce6e34841c090
Oct 30 21:36:25.999687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:25.999663    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"]
Oct 30 21:36:26.001831 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:26.001799    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:26.001924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:26.001874    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:26.001924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:26.001918    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:26.734701 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:26.734657    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" event={"ID":"0910c2f7-ba5e-48fe-baaa-087201160f94","Type":"ContainerStarted","Data":"1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24"}
Oct 30 21:36:26.734877 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:26.734726    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" event={"ID":"0910c2f7-ba5e-48fe-baaa-087201160f94","Type":"ContainerStarted","Data":"2ec17d0e698cd1662a3e20accfcc25d5f9796d0411043009080ce6e34841c090"}
Oct 30 21:36:26.885011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:26.884939    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podStartSLOduration=2.884916737 podStartE2EDuration="2.884916737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:26.873129059 +0000 UTC m=+1442.744248180" watchObservedRunningTime="2026-10-30 21:36:26.884916737 +0000 UTC m=+1442.756035856"
Oct 30 21:36:26.904123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:26.900805    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:36:26.904123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:26.900900    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:36:27.537171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:27.537136    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:27.538514 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:27.538484    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 21:36:27.538618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:27.538543    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 21:36:28.228708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:28.228679    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:28.229271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:28.229244    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:28.230095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:28.230078    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:36:28.537189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:28.537116    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 21:36:28.537189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:28.537169    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 21:36:29.539232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:29.538856    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 21:36:29.539232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:29.538931    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 21:36:30.538236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:30.537561    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 21:36:30.538236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:30.537640    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 21:36:30.773234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:30.756278    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:36:31.548232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:31.539634    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 21:36:31.548232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:31.539728    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 21:36:31.578232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:31.576372    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:31.664402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:31.664358    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:31.708404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:31.708355    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/health\": context deadline exceeded"
Oct 30 21:36:31.753959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:31.753921    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 21:36:31.797344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:31.797303    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:36:31.797938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:31.797907    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.059693 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.059650    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.061086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.059940    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.069237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.069072    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"]
Oct 30 21:36:32.101229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.097409    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.159306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.140291    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-metrics-certs\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.159306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.140428    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.159306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.154373    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667d7d7d-53bd-4597-8def-7c5025fc636c-openshift-service-ca\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.159306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.154459    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgs6h\" (UniqueName: \"kubernetes.io/projected/667d7d7d-53bd-4597-8def-7c5025fc636c-kube-api-access-vgs6h\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.159306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.154678    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.159306 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.155038    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.162073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.161229    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6f787654-serving-cert\""
Oct 30 21:36:32.187133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.177286    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.232231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.206937    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.233275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.223828    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.233928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.233852    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"]
Oct 30 21:36:32.291231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.262294    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.291231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.264516    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667d7d7d-53bd-4597-8def-7c5025fc636c-openshift-service-ca\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.291231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.264611    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgs6h\" (UniqueName: \"kubernetes.io/projected/667d7d7d-53bd-4597-8def-7c5025fc636c-kube-api-access-vgs6h\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.291231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.264873    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-metrics-certs\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.291231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.265005    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.291231 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:32.265165    5786 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-serving-cert: secret "maas-api-e2e-derive-6f787654-serving-cert" not found
Oct 30 21:36:32.291231 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:32.265244    5786 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls podName:667d7d7d-53bd-4597-8def-7c5025fc636c nodeName:}" failed. No retries permitted until 2026-10-30 21:36:32.765223157 +0000 UTC m=+1448.636342276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls") pod "maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" (UID: "667d7d7d-53bd-4597-8def-7c5025fc636c") : secret "maas-api-e2e-derive-6f787654-serving-cert" not found
Oct 30 21:36:32.291231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.265281    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:36:32.299236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.292414    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667d7d7d-53bd-4597-8def-7c5025fc636c-openshift-service-ca\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.314292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.306560    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:36:32.484262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.445924    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgs6h\" (UniqueName: \"kubernetes.io/projected/667d7d7d-53bd-4597-8def-7c5025fc636c-kube-api-access-vgs6h\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.667291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.666933    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-metrics-certs\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:32.743591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.724525    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 21:36:32.743591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.724604    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 21:36:32.782267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.755462    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:32.951290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:32.946360    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:33.082432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.082372    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": context deadline exceeded"
Oct 30 21:36:33.200273 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.131278    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:33.255329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.168459    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:33.260299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.168897    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:33.265218 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.264916    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:33.265676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.265482    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:33.318950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.266041    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:36:33.402243 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.357518    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:33.428230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.360537    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:36:33.428230 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:33.401013    5786 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.035s"
Oct 30 21:36:33.428230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.416125    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:36:33.428230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.420285    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:36:33.428230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.420492    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:36:33.428230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.420579    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:36:33.428230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.424426    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:36:33.428722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.428406    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:36:33.438229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.432336    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:36:33.438229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.432424    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:36:33.438229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.436317    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:36:33.438229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.436412    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:36:33.607286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.486427    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:33.607286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.513410    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:36:33.672275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.646953    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:36:33.672275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.665252    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 21:36:33.672275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.665309    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 21:36:33.726595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.722985    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:36:33.726595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.723053    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:33.776229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.775509    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 21:36:33.971323 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:33.971278    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 21:36:34.054415 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.054380    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls\") pod \"maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:34.238002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.237916    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs" podUID="6523484b-480f-4c20-b123-8013f0ef711e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": context deadline exceeded"
Oct 30 21:36:34.255353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.255310    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 30 21:36:34.255677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.255649    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:36:34.331419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.331290    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:36:34.333885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.333478    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:34.367260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.356584    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:34.497110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.497019    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:34.602232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.593346    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" probeResult="failure" output="Get \"https://10.134.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:34.703281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.603975    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:34.801282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.659789    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:36:34.801282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.793063    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs"
Oct 30 21:36:34.801282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.793103    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:36:34.801282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.774457    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 21:36:34.801282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.793158    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 21:36:34.886256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.881248    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:36:34.920444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.920412    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:36:34.946969 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.946925    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:36:34.947130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.947071    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:36:34.947130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.947106    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:36:34.947130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:34.947122    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"]
Oct 30 21:36:35.046498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.046469    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"]
Oct 30 21:36:35.047129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.047106    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.074499    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.074599    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f3b286-fd29-4721-92c1-3d65435a48d4-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.074691    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.074730    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.074753    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.074808    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.074992    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlmmq\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-kube-api-access-qlmmq\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.075054    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.075860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.075123    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180035    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlmmq\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-kube-api-access-qlmmq\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180137    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180260    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180324    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180462    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f3b286-fd29-4721-92c1-3d65435a48d4-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180568    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180626    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180752 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180652    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.180752 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.180713    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.186278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.184594    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-data\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.186278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.185450    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-credential-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.189471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.188898    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-podinfo\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.190643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.190619    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f3b286-fd29-4721-92c1-3d65435a48d4-istiod-ca-cert\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.191612 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.191583    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-socket\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.193323 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.193303    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-certs\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.199862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.199820    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-envoy\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.239523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.239493    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlmmq\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-kube-api-access-qlmmq\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.246575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.246544    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-token\") pod \"e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") " pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.424887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.424837    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:35.680402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.680337    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"]
Oct 30 21:36:35.996261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.996198    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:35.996521 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.996494    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:35.996661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:35.996639    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:36.026977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:36.026950    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"]
Oct 30 21:36:36.160249 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:36.159520    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:36.161383 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:36.160851    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:36.161902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:36.161888    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:36:36.174476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:36.174446    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" event={"ID":"667d7d7d-53bd-4597-8def-7c5025fc636c","Type":"ContainerStarted","Data":"97a01b776b0b291af936aa27d5345676afb75048e99d454c239d427d5f1923bf"}
Oct 30 21:36:36.176993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:36.176966    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" event={"ID":"99f3b286-fd29-4721-92c1-3d65435a48d4","Type":"ContainerStarted","Data":"b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c"}
Oct 30 21:36:36.177142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:36.177019    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" event={"ID":"99f3b286-fd29-4721-92c1-3d65435a48d4","Type":"ContainerStarted","Data":"0c9f2843b67255b34182f7ec0f9ed1c370d8ebef095ac89b988ed517a7129b35"}
Oct 30 21:36:36.428009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:36.427978    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:37.215538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:37.215494    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" event={"ID":"667d7d7d-53bd-4597-8def-7c5025fc636c","Type":"ContainerStarted","Data":"b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec"}
Oct 30 21:36:37.216254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:37.216229    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:37.345749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:37.345704    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" podStartSLOduration=3.345690592 podStartE2EDuration="3.345690592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:36.974027049 +0000 UTC m=+1452.845146168" watchObservedRunningTime="2026-10-30 21:36:37.345690592 +0000 UTC m=+1453.216809715"
Oct 30 21:36:37.346359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:37.346334    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" podStartSLOduration=5.386164213 podStartE2EDuration="6.346327772s" totalImagesPullingTime="960.163559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:37.340837784 +0000 UTC m=+1453.211956902" watchObservedRunningTime="2026-10-30 21:36:37.346327772 +0000 UTC m=+1453.217446916"
Oct 30 21:36:37.429896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:37.429867    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:36:37.430048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:37.429917    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:36:38.428246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:38.428199    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:36:38.428722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:38.428280    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:36:39.428281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:39.428246    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:36:39.428676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:39.428305    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:36:40.428957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:40.428922    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:36:40.429338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:40.428981    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:36:41.428170 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:41.428138    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:36:41.428382 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:41.428243    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:36:42.434651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:42.434619    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:36:42.435258 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:42.435223    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:36:43.433449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:43.433421    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:43.433955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:43.433933    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:43.434772 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:43.434752    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:36:44.234097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:44.234062    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.139:8443/health\": dial tcp 10.134.0.139:8443: connect: connection refused"
Oct 30 21:36:47.309338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:47.309301    5786 generic.go:356] "Generic (PLEG): container finished" podID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerID="b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec" exitCode=1
Oct 30 21:36:47.309816 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:47.309368    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" event={"ID":"667d7d7d-53bd-4597-8def-7c5025fc636c","Type":"ContainerDied","Data":"b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec"}
Oct 30 21:36:47.309816 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:47.309790    5786 scope.go:122] "RemoveContainer" containerID="b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec"
Oct 30 21:36:48.319084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:48.319050    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" event={"ID":"667d7d7d-53bd-4597-8def-7c5025fc636c","Type":"ContainerStarted","Data":"c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df"}
Oct 30 21:36:48.319559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:48.319467    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:52.357226 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:36:52.357171    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:36:54.331727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:54.331705    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:36:55.366701 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.366671    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"]
Oct 30 21:36:55.372663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.372639    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443175 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443139    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6567df37-bf65-475c-bc04-d7f9c68667ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443253    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txgkn\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-kube-api-access-txgkn\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443369    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443404    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443537    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443599    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443713    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443765    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.443921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.443900    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.544896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.544864    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545036    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6567df37-bf65-475c-bc04-d7f9c68667ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545066    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txgkn\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-kube-api-access-txgkn\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545127    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545154    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545249    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545371    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545445    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545487    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545512    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545540    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545685    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545757    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6567df37-bf65-475c-bc04-d7f9c68667ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.545835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.545762    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.547879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.547844    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.548018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.548003    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.588907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.588877    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"]
Oct 30 21:36:55.601974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.601951    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.602342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.602314    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txgkn\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-kube-api-access-txgkn\") pod \"maas-default-gateway-openshift-default-6cc458d887-v64g9\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:55.687751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:55.687725    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:56.124533 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:36:56.124501    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6567df37_bf65_475c_bc04_d7f9c68667ad.slice/crio-64e5323e9b1382a28c3385ede05f349059e54571776d97e829821a01b7b75c15 WatchSource:0}: Error finding container 64e5323e9b1382a28c3385ede05f349059e54571776d97e829821a01b7b75c15: Status 404 returned error can't find the container with id 64e5323e9b1382a28c3385ede05f349059e54571776d97e829821a01b7b75c15
Oct 30 21:36:56.124873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:56.124855    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"]
Oct 30 21:36:56.127270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:56.127237    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:56.127392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:56.127317    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:56.127392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:56.127356    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:36:56.388335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:56.388247    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" event={"ID":"6567df37-bf65-475c-bc04-d7f9c68667ad","Type":"ContainerStarted","Data":"8f315b550478de98765c2ac8428262608eb78ad8bad9251acf2c5e824e73541c"}
Oct 30 21:36:56.388335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:56.388304    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" event={"ID":"6567df37-bf65-475c-bc04-d7f9c68667ad","Type":"ContainerStarted","Data":"64e5323e9b1382a28c3385ede05f349059e54571776d97e829821a01b7b75c15"}
Oct 30 21:36:56.507694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:56.507638    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podStartSLOduration=1.507619661 podStartE2EDuration="1.507619661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:56.503830532 +0000 UTC m=+1472.374949652" watchObservedRunningTime="2026-10-30 21:36:56.507619661 +0000 UTC m=+1472.378738782"
Oct 30 21:36:56.687921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:56.687888    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:36:57.689836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:57.689801    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v64g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:36:57.690482 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:57.689864    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:36:58.688962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:58.688926    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v64g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:36:58.689121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:58.688991    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:36:59.688793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:59.688760    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v64g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:36:59.689322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:36:59.688816    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:37:00.688691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:00.688658    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v64g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:37:00.688881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:00.688715    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:37:01.688046 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:01.688011    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v64g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:37:01.688256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:01.688068    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:37:02.688807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:02.688776    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v64g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:37:02.689220 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:02.688824    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:37:04.417337 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:04.417300    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:37:04.418061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:04.418027    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:37:04.418859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:04.418841    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:37:22.392999 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:37:22.392965    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:37:25.083962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:25.081816    5786 patch_prober.go:29] interesting pod/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:37:25.083962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:25.081892    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:27.118762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:27.118725    5786 generic.go:356] "Generic (PLEG): container finished" podID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerID="fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151" exitCode=0
Oct 30 21:37:27.119188 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:27.118803    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" event={"ID":"411e28c0-bd7b-4294-b39f-0cb678be0fbf","Type":"ContainerDied","Data":"fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151"}
Oct 30 21:37:27.119429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:27.119412    5786 scope.go:122] "RemoveContainer" containerID="fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151"
Oct 30 21:37:27.121691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:27.121655    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:27.121791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:27.121746    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:27.121832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:27.121792    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:28.000385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:28.000309    5786 patch_prober.go:29] interesting pod/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:37:28.000385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:28.000373    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:28.298106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:28.129244    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" event={"ID":"411e28c0-bd7b-4294-b39f-0cb678be0fbf","Type":"ContainerStarted","Data":"f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1"}
Oct 30 21:37:28.612709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:28.612685    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:37:28.612899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:28.612877    5786 patch_prober.go:29] interesting pod/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:37:28.612968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:28.612929    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 21:37:29.070535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.070501    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"]
Oct 30 21:37:29.100828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.100798    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8efcd01-ctw49"]
Oct 30 21:37:29.138909 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.138887    5786 generic.go:356] "Generic (PLEG): container finished" podID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerID="c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f" exitCode=0
Oct 30 21:37:29.139049 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.138962    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" event={"ID":"c65c2976-ed99-4599-93cd-c1fa38b5df34","Type":"ContainerDied","Data":"c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f"}
Oct 30 21:37:29.139468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.139449    5786 scope.go:122] "RemoveContainer" containerID="c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f"
Oct 30 21:37:29.140833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.140814    5786 generic.go:356] "Generic (PLEG): container finished" podID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerID="f9c905dac5f0dd5dd12dd21645e9b0d14acf80b7e528b2bfab47291ad8beec26" exitCode=0
Oct 30 21:37:29.140930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.140910    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" event={"ID":"dcf64b51-cf8b-47ca-81e6-d9365f1de030","Type":"ContainerDied","Data":"f9c905dac5f0dd5dd12dd21645e9b0d14acf80b7e528b2bfab47291ad8beec26"}
Oct 30 21:37:29.141053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.141029    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:29.141106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.141093    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:29.141159 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.141123    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:29.141251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.141228    5786 scope.go:122] "RemoveContainer" containerID="f9c905dac5f0dd5dd12dd21645e9b0d14acf80b7e528b2bfab47291ad8beec26"
Oct 30 21:37:29.142718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.142692    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:29.142821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.142750    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:29.142821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.142783    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:37:29.613769 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.613728    5786 patch_prober.go:29] interesting pod/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:37:29.614305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:29.613776    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 21:37:30.166243 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:30.164356    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" event={"ID":"c65c2976-ed99-4599-93cd-c1fa38b5df34","Type":"ContainerStarted","Data":"f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65"}
Oct 30 21:37:30.198973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:30.198937    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" event={"ID":"dcf64b51-cf8b-47ca-81e6-d9365f1de030","Type":"ContainerStarted","Data":"53073a71031e400ace45d637f92a08b48e2555e0905c085806a0a2ce91588921"}
Oct 30 21:37:30.447076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:30.443615    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="632fb0ea-ea0b-4b62-a841-e55f940c3da0" path="/var/lib/kubelet/pods/632fb0ea-ea0b-4b62-a841-e55f940c3da0/volumes"
Oct 30 21:37:30.584233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:30.581679    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:37:30.619227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:30.619138    5786 patch_prober.go:29] interesting pod/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:37:30.619227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:30.619193    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 21:37:30.619806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:30.619628    5786 patch_prober.go:29] interesting pod/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:37:30.619806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:30.619667    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 21:37:31.186236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:31.186103    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:37:31.186449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:31.186327    5786 patch_prober.go:29] interesting pod/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:37:31.186449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:31.186380    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 21:37:31.577272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:31.577166    5786 patch_prober.go:29] interesting pod/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:37:31.577531 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:31.577505    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 21:37:31.619652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:31.619613    5786 patch_prober.go:29] interesting pod/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:37:31.620340 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:31.620275    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 21:37:32.180796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:32.180751    5786 patch_prober.go:29] interesting pod/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:37:32.181119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:32.181075    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 21:37:32.583309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:32.574103    5786 patch_prober.go:29] interesting pod/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:37:32.583309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:32.574161    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 21:37:32.618762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:32.618647    5786 patch_prober.go:29] interesting pod/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:37:32.618762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:32.618704    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 21:37:33.194443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:33.194408    5786 patch_prober.go:29] interesting pod/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:37:33.195041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:33.195009    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 21:37:33.245742 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:33.245707    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:37:33.573743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:33.573655    5786 patch_prober.go:29] interesting pod/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:37:33.573993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:33.573966    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 21:37:33.625711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:33.625577    5786 patch_prober.go:29] interesting pod/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:37:33.625711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:33.625650    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 21:37:34.190322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:34.181579    5786 patch_prober.go:29] interesting pod/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:37:34.190322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:34.181645    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 21:37:34.582401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:34.582312    5786 patch_prober.go:29] interesting pod/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:37:34.582959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:34.582928    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 21:37:34.616849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:34.616824    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:37:34.617123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:34.617103    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:37:34.618015 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:34.617996    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:37:35.189640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:35.189591    5786 patch_prober.go:29] interesting pod/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:37:35.189927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:35.189898    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 21:37:35.635352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:35.628604    5786 patch_prober.go:29] interesting pod/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:37:35.638092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:35.638054    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 21:37:36.204236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:36.192310    5786 patch_prober.go:29] interesting pod/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:37:36.204236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:36.192371    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 21:37:36.581990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:36.581892    5786 patch_prober.go:29] interesting pod/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:37:36.582308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:36.582265    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 21:37:37.180698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:37.180662    5786 patch_prober.go:29] interesting pod/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:37:37.181064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:37.180721    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 21:37:37.575305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:37.575236    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:37:37.575646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:37.575618    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:37:37.576169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:37.576152    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:37:38.184437 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:38.184411    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:37:38.185545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:38.185523    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:37:40.055012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:40.054981    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"]
Oct 30 21:37:40.103607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:40.103577    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b473c34a-5dpw8"]
Oct 30 21:37:40.362235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:40.362149    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f8e7f3e-bc12-4062-82df-cc85bd0f7d44" path="/var/lib/kubelet/pods/8f8e7f3e-bc12-4062-82df-cc85bd0f7d44/volumes"
Oct 30 21:37:49.587763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:49.587731    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"]
Oct 30 21:37:52.321981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:52.321946    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"]
Oct 30 21:37:52.355594 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:37:52.355556    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:37:54.588038 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:54.588003    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf" podUID="a192c0be-4988-4876-a080-a890a9f360e8" containerName="payload-processing" containerID="cri-o://6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b" gracePeriod=25
Oct 30 21:37:54.871272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:54.871249    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:37:54.997928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:54.997894    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a192c0be-4988-4876-a080-a890a9f360e8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a192c0be-4988-4876-a080-a890a9f360e8-plugins-config-volume\") pod \"a192c0be-4988-4876-a080-a890a9f360e8\" (UID: \"a192c0be-4988-4876-a080-a890a9f360e8\") "
Oct 30 21:37:54.998083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:54.997952    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a192c0be-4988-4876-a080-a890a9f360e8-kube-api-access-f5fpv\" (UniqueName: \"kubernetes.io/projected/a192c0be-4988-4876-a080-a890a9f360e8-kube-api-access-f5fpv\") pod \"a192c0be-4988-4876-a080-a890a9f360e8\" (UID: \"a192c0be-4988-4876-a080-a890a9f360e8\") "
Oct 30 21:37:54.998394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:54.998361    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a192c0be-4988-4876-a080-a890a9f360e8-plugins-config-volume" pod "a192c0be-4988-4876-a080-a890a9f360e8" (UID: "a192c0be-4988-4876-a080-a890a9f360e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:54.998764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:54.998740    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a192c0be-4988-4876-a080-a890a9f360e8-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:55.000193 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.000170    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a192c0be-4988-4876-a080-a890a9f360e8-kube-api-access-f5fpv" pod "a192c0be-4988-4876-a080-a890a9f360e8" (UID: "a192c0be-4988-4876-a080-a890a9f360e8"). InnerVolumeSpecName "kube-api-access-f5fpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:55.100841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.100780    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5fpv\" (UniqueName: \"kubernetes.io/projected/a192c0be-4988-4876-a080-a890a9f360e8-kube-api-access-f5fpv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:55.628112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.628083    5786 generic.go:356] "Generic (PLEG): container finished" podID="a192c0be-4988-4876-a080-a890a9f360e8" containerID="6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b" exitCode=0
Oct 30 21:37:55.628609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.628127    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"
Oct 30 21:37:55.628609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.628163    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf" event={"ID":"a192c0be-4988-4876-a080-a890a9f360e8","Type":"ContainerDied","Data":"6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b"}
Oct 30 21:37:55.628609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.628256    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf" event={"ID":"a192c0be-4988-4876-a080-a890a9f360e8","Type":"ContainerDied","Data":"9bff5d1b068f8c69c4a58a4595c979e2faa2889212f5c58ad2fcafccd246a9c8"}
Oct 30 21:37:55.628609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.628290    5786 scope.go:122] "RemoveContainer" containerID="6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b"
Oct 30 21:37:55.639327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.639311    5786 scope.go:122] "RemoveContainer" containerID="6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b"
Oct 30 21:37:55.639569 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:37:55.639551    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b\": container with ID starting with 6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b not found: ID does not exist" containerID="6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b"
Oct 30 21:37:55.639635 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.639575    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b"} err="failed to get container status \"6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b\": rpc error: code = NotFound desc = could not find container \"6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b\": container with ID starting with 6952f201d793e871780ab0f6fc02d808b94743173bf061e260554d5dc608265b not found: ID does not exist"
Oct 30 21:37:55.794759 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.794734    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"]
Oct 30 21:37:55.799960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:55.799922    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d5266a5-78799466bc-kxvsf"]
Oct 30 21:37:56.361117 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:56.361086    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a192c0be-4988-4876-a080-a890a9f360e8" path="/var/lib/kubelet/pods/a192c0be-4988-4876-a080-a890a9f360e8/volumes"
Oct 30 21:37:57.322604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.322565    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc" podUID="8d1c57f4-80c3-4c3c-b679-de2945760f76" containerName="payload-pre-processing" containerID="cri-o://71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034" gracePeriod=25
Oct 30 21:37:57.617072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.617051    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:37:57.646521 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.646494    5786 generic.go:356] "Generic (PLEG): container finished" podID="8d1c57f4-80c3-4c3c-b679-de2945760f76" containerID="71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034" exitCode=0
Oct 30 21:37:57.646654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.646536    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"
Oct 30 21:37:57.646654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.646580    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc" event={"ID":"8d1c57f4-80c3-4c3c-b679-de2945760f76","Type":"ContainerDied","Data":"71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034"}
Oct 30 21:37:57.646654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.646638    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc" event={"ID":"8d1c57f4-80c3-4c3c-b679-de2945760f76","Type":"ContainerDied","Data":"de157b633aed310580e7898ab70593e44409d6d0b98751be963b130ebe401729"}
Oct 30 21:37:57.646815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.646668    5786 scope.go:122] "RemoveContainer" containerID="71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034"
Oct 30 21:37:57.659771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.659695    5786 scope.go:122] "RemoveContainer" containerID="71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034"
Oct 30 21:37:57.660786 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:37:57.660755    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034\": container with ID starting with 71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034 not found: ID does not exist" containerID="71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034"
Oct 30 21:37:57.660902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.660781    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034"} err="failed to get container status \"71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034\": rpc error: code = NotFound desc = could not find container \"71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034\": container with ID starting with 71789f5edde8ac24f77eb50d876ddeca0b93e90b5a2f840f5a7590887f7bb034 not found: ID does not exist"
Oct 30 21:37:57.742887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.742852    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d1c57f4-80c3-4c3c-b679-de2945760f76-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d1c57f4-80c3-4c3c-b679-de2945760f76-plugins-config-volume\") pod \"8d1c57f4-80c3-4c3c-b679-de2945760f76\" (UID: \"8d1c57f4-80c3-4c3c-b679-de2945760f76\") "
Oct 30 21:37:57.743141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.742916    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d1c57f4-80c3-4c3c-b679-de2945760f76-kube-api-access-j2rr9\" (UniqueName: \"kubernetes.io/projected/8d1c57f4-80c3-4c3c-b679-de2945760f76-kube-api-access-j2rr9\") pod \"8d1c57f4-80c3-4c3c-b679-de2945760f76\" (UID: \"8d1c57f4-80c3-4c3c-b679-de2945760f76\") "
Oct 30 21:37:57.743654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.743625    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d1c57f4-80c3-4c3c-b679-de2945760f76-plugins-config-volume" pod "8d1c57f4-80c3-4c3c-b679-de2945760f76" (UID: "8d1c57f4-80c3-4c3c-b679-de2945760f76"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:57.753234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.752362    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d1c57f4-80c3-4c3c-b679-de2945760f76-kube-api-access-j2rr9" pod "8d1c57f4-80c3-4c3c-b679-de2945760f76" (UID: "8d1c57f4-80c3-4c3c-b679-de2945760f76"). InnerVolumeSpecName "kube-api-access-j2rr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:57.766505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.766477    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d1c57f4-80c3-4c3c-b679-de2945760f76-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:57.766604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:57.766507    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2rr9\" (UniqueName: \"kubernetes.io/projected/8d1c57f4-80c3-4c3c-b679-de2945760f76-kube-api-access-j2rr9\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:58.053896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:58.053868    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"]
Oct 30 21:37:58.060543 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:58.060511    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-699d67f6-snvn5"]
Oct 30 21:37:58.069652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:58.069627    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"]
Oct 30 21:37:58.073504 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:58.073487    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f787654-545f879d87-n2kdc"]
Oct 30 21:37:58.361927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:58.361850    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d1c57f4-80c3-4c3c-b679-de2945760f76" path="/var/lib/kubelet/pods/8d1c57f4-80c3-4c3c-b679-de2945760f76/volumes"
Oct 30 21:37:58.362574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:37:58.362555    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b84821b3-6773-419e-9634-f6f6e1f95f03" path="/var/lib/kubelet/pods/b84821b3-6773-419e-9634-f6f6e1f95f03/volumes"
Oct 30 21:38:09.483718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:09.483527    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"]
Oct 30 21:38:09.484153 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:09.483750    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f" podUID="1b56cbbb-7ddd-46ea-a284-ff469f5240af" containerName="maas-api" containerID="cri-o://0aa91506720d587defcd4d7bd0174442d6a6bf63e114ca36b5f8bd4eb9959732" gracePeriod=30
Oct 30 21:38:09.964424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:09.963993    5786 generic.go:356] "Generic (PLEG): container finished" podID="1b56cbbb-7ddd-46ea-a284-ff469f5240af" containerID="0aa91506720d587defcd4d7bd0174442d6a6bf63e114ca36b5f8bd4eb9959732" exitCode=0
Oct 30 21:38:09.964424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:09.964319    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f" event={"ID":"1b56cbbb-7ddd-46ea-a284-ff469f5240af","Type":"ContainerDied","Data":"0aa91506720d587defcd4d7bd0174442d6a6bf63e114ca36b5f8bd4eb9959732"}
Oct 30 21:38:10.263224 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.260529    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:38:10.414278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.411646    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b56cbbb-7ddd-46ea-a284-ff469f5240af-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b56cbbb-7ddd-46ea-a284-ff469f5240af-openshift-service-ca\") pod \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") "
Oct 30 21:38:10.414278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.411887    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b56cbbb-7ddd-46ea-a284-ff469f5240af-kube-api-access-zwckj\" (UniqueName: \"kubernetes.io/projected/1b56cbbb-7ddd-46ea-a284-ff469f5240af-kube-api-access-zwckj\") pod \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") "
Oct 30 21:38:10.414278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.411959    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls\") pod \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") "
Oct 30 21:38:10.414278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.412079    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-metrics-certs\") pod \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\" (UID: \"1b56cbbb-7ddd-46ea-a284-ff469f5240af\") "
Oct 30 21:38:10.419471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.419393    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b56cbbb-7ddd-46ea-a284-ff469f5240af-openshift-service-ca" pod "1b56cbbb-7ddd-46ea-a284-ff469f5240af" (UID: "1b56cbbb-7ddd-46ea-a284-ff469f5240af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:10.432302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.427660    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b56cbbb-7ddd-46ea-a284-ff469f5240af-kube-api-access-zwckj" pod "1b56cbbb-7ddd-46ea-a284-ff469f5240af" (UID: "1b56cbbb-7ddd-46ea-a284-ff469f5240af"). InnerVolumeSpecName "kube-api-access-zwckj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:10.432302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.430710    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-metrics-certs" pod "1b56cbbb-7ddd-46ea-a284-ff469f5240af" (UID: "1b56cbbb-7ddd-46ea-a284-ff469f5240af"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:10.432302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.431070    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls" pod "1b56cbbb-7ddd-46ea-a284-ff469f5240af" (UID: "1b56cbbb-7ddd-46ea-a284-ff469f5240af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:10.434100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.434075    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b56cbbb-7ddd-46ea-a284-ff469f5240af-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:10.435774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.435755    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwckj\" (UniqueName: \"kubernetes.io/projected/1b56cbbb-7ddd-46ea-a284-ff469f5240af-kube-api-access-zwckj\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:10.435930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.435915    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:10.436019 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:10.436008    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b56cbbb-7ddd-46ea-a284-ff469f5240af-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:11.013116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:11.001970    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f" event={"ID":"1b56cbbb-7ddd-46ea-a284-ff469f5240af","Type":"ContainerDied","Data":"9f9106bfac90753f04cf9a2e9c443ff9b75d95408ca404493e0da52719fe4db2"}
Oct 30 21:38:11.013116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:11.002077    5786 scope.go:122] "RemoveContainer" containerID="0aa91506720d587defcd4d7bd0174442d6a6bf63e114ca36b5f8bd4eb9959732"
Oct 30 21:38:11.013116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:11.002313    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"
Oct 30 21:38:11.172813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:11.172779    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"]
Oct 30 21:38:11.221373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:11.221331    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b25d536b-6b68c447d8-lf29f"]
Oct 30 21:38:11.771310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:11.771279    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"]
Oct 30 21:38:11.771564 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:11.771538    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy" containerID="cri-o://f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1" gracePeriod=30
Oct 30 21:38:12.361414 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:12.361377    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b56cbbb-7ddd-46ea-a284-ff469f5240af" path="/var/lib/kubelet/pods/1b56cbbb-7ddd-46ea-a284-ff469f5240af/volumes"
Oct 30 21:38:13.981339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.981310    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"]
Oct 30 21:38:13.982069 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982053    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a192c0be-4988-4876-a080-a890a9f360e8" containerName="payload-processing"
Oct 30 21:38:13.982128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982071    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a192c0be-4988-4876-a080-a890a9f360e8" containerName="payload-processing"
Oct 30 21:38:13.982128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982089    5786 cpu_manager.go:414] "containerMap: removing container" podUID="1b56cbbb-7ddd-46ea-a284-ff469f5240af" containerName="maas-api"
Oct 30 21:38:13.982128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982097    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b56cbbb-7ddd-46ea-a284-ff469f5240af" containerName="maas-api"
Oct 30 21:38:13.982262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982142    5786 cpu_manager.go:414] "containerMap: removing container" podUID="8d1c57f4-80c3-4c3c-b679-de2945760f76" containerName="payload-pre-processing"
Oct 30 21:38:13.982262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982147    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d1c57f4-80c3-4c3c-b679-de2945760f76" containerName="payload-pre-processing"
Oct 30 21:38:13.982328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982301    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a192c0be-4988-4876-a080-a890a9f360e8" containerName="payload-processing"
Oct 30 21:38:13.982328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982312    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b56cbbb-7ddd-46ea-a284-ff469f5240af" containerName="maas-api"
Oct 30 21:38:13.982384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.982332    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d1c57f4-80c3-4c3c-b679-de2945760f76" containerName="payload-pre-processing"
Oct 30 21:38:13.985573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:13.985557    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.037317 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.037293    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-a7a6ff99-gw-openshift-default-dockercfg-8vw5x\""
Oct 30 21:38:14.092038 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.092009    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"]
Oct 30 21:38:14.097257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.097231    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bd9d54e5-lj8s2"]
Oct 30 21:38:14.103065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.103038    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"]
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175523    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-envoy\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175617    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-podinfo\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175668    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-token\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175730    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-socket\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175811    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzsc8\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-kube-api-access-wzsc8\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175847    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-credential-socket\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175883    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-certs\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175916    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b39fbcd0-dd33-4041-8496-c9310c19856e-istiod-ca-cert\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.177782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.175984    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-data\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278288    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-envoy\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278362    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-podinfo\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278406    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-token\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278456    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-socket\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278517    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzsc8\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-kube-api-access-wzsc8\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278544    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-credential-socket\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278580    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-certs\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278616    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b39fbcd0-dd33-4041-8496-c9310c19856e-istiod-ca-cert\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.278673    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-data\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.279151    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-data\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.289990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.289874    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-certs\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.301119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.290128    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-credential-socket\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.301119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.291121    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b39fbcd0-dd33-4041-8496-c9310c19856e-istiod-ca-cert\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.301119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.295438    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-socket\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.301119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.299222    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-envoy\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.301119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.300916    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-podinfo\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.394733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.375661    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-token\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.396193 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.396106    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzsc8\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-kube-api-access-wzsc8\") pod \"e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") " pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:14.402885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.402862    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="284e5e57-72e5-48a5-b746-3eb81504ff63" path="/var/lib/kubelet/pods/284e5e57-72e5-48a5-b746-3eb81504ff63/volumes"
Oct 30 21:38:14.599322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:14.599266    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:15.036967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:15.036935    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"]
Oct 30 21:38:15.037366 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:38:15.037129    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39fbcd0_dd33_4041_8496_c9310c19856e.slice/crio-24c0665d2230732f959a670e14fd6d34b0b41665dabd1fb55a852b111cad658e WatchSource:0}: Error finding container 24c0665d2230732f959a670e14fd6d34b0b41665dabd1fb55a852b111cad658e: Status 404 returned error can't find the container with id 24c0665d2230732f959a670e14fd6d34b0b41665dabd1fb55a852b111cad658e
Oct 30 21:38:15.039511 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:15.039479    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:15.039600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:15.039544    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:15.039600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:15.039574    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:15.093078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:15.093050    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" event={"ID":"b39fbcd0-dd33-4041-8496-c9310c19856e","Type":"ContainerStarted","Data":"24c0665d2230732f959a670e14fd6d34b0b41665dabd1fb55a852b111cad658e"}
Oct 30 21:38:15.760457 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:15.760425    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"]
Oct 30 21:38:15.760688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:15.760665    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm" podUID="c7ae7dd9-4560-4a29-8425-0acc03ff57d5" containerName="maas-api" containerID="cri-o://a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306" gracePeriod=30
Oct 30 21:38:16.158643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.158618    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:38:16.241733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.237660    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-metrics-certs\") pod \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") "
Oct 30 21:38:16.241733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.237861    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-kube-api-access-5h4zs\" (UniqueName: \"kubernetes.io/projected/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-kube-api-access-5h4zs\") pod \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") "
Oct 30 21:38:16.241733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.238026    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-openshift-service-ca\") pod \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") "
Oct 30 21:38:16.262266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.246757    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls\") pod \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\" (UID: \"c7ae7dd9-4560-4a29-8425-0acc03ff57d5\") "
Oct 30 21:38:16.285984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.267095    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-openshift-service-ca" pod "c7ae7dd9-4560-4a29-8425-0acc03ff57d5" (UID: "c7ae7dd9-4560-4a29-8425-0acc03ff57d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:16.312234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.301821    5786 generic.go:356] "Generic (PLEG): container finished" podID="c7ae7dd9-4560-4a29-8425-0acc03ff57d5" containerID="a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306" exitCode=0
Oct 30 21:38:16.312234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.302039    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm" event={"ID":"c7ae7dd9-4560-4a29-8425-0acc03ff57d5","Type":"ContainerDied","Data":"a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306"}
Oct 30 21:38:16.312234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.302087    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm" event={"ID":"c7ae7dd9-4560-4a29-8425-0acc03ff57d5","Type":"ContainerDied","Data":"e853248f10afc173bf5e5bd7218f01d30ca4675ab1e47fe5f0e7e99d2166d00c"}
Oct 30 21:38:16.312234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.302114    5786 scope.go:122] "RemoveContainer" containerID="a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306"
Oct 30 21:38:16.312234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.302264    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"
Oct 30 21:38:16.328524 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.323446    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-kube-api-access-5h4zs" pod "c7ae7dd9-4560-4a29-8425-0acc03ff57d5" (UID: "c7ae7dd9-4560-4a29-8425-0acc03ff57d5"). InnerVolumeSpecName "kube-api-access-5h4zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:16.363477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.363368    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:16.363705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.363685    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h4zs\" (UniqueName: \"kubernetes.io/projected/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-kube-api-access-5h4zs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:16.444999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.444935    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-metrics-certs" pod "c7ae7dd9-4560-4a29-8425-0acc03ff57d5" (UID: "c7ae7dd9-4560-4a29-8425-0acc03ff57d5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:16.448986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.448920    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls" pod "c7ae7dd9-4560-4a29-8425-0acc03ff57d5" (UID: "c7ae7dd9-4560-4a29-8425-0acc03ff57d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:16.466454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.465787    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:16.466454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.465815    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7ae7dd9-4560-4a29-8425-0acc03ff57d5-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:16.479054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.479016    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" event={"ID":"b39fbcd0-dd33-4041-8496-c9310c19856e","Type":"ContainerStarted","Data":"ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e"}
Oct 30 21:38:16.481354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.481335    5786 scope.go:122] "RemoveContainer" containerID="a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306"
Oct 30 21:38:16.481675 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:16.481648    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306\": container with ID starting with a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306 not found: ID does not exist" containerID="a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306"
Oct 30 21:38:16.481803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.481680    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306"} err="failed to get container status \"a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306\": rpc error: code = NotFound desc = could not find container \"a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306\": container with ID starting with a33f164512ce1304a145c9c4dd497471e9a801d0cb458fa7c02e879040a44306 not found: ID does not exist"
Oct 30 21:38:16.548537 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:16.548511    5786 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ae7dd9_4560_4a29_8425_0acc03ff57d5.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ae7dd9_4560_4a29_8425_0acc03ff57d5.slice/crio-e853248f10afc173bf5e5bd7218f01d30ca4675ab1e47fe5f0e7e99d2166d00c\": RecentStats: unable to find data in memory cache]"
Oct 30 21:38:16.587839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.587798    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" podStartSLOduration=3.5877859560000003 podStartE2EDuration="3.587785956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:16.585759174 +0000 UTC m=+1552.456878293" watchObservedRunningTime="2026-10-30 21:38:16.587785956 +0000 UTC m=+1552.458905074"
Oct 30 21:38:16.599766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.599745    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:16.605035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.604996    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:16.910237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.900283    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"]
Oct 30 21:38:16.922526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:16.922485    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d5266a5-68f86cd84f-km9hm"]
Oct 30 21:38:17.151102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.151082    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:38:17.312903 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.312870    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-socket\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.313392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.312999    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-credential-socket\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.313392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313110    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-socket" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:17.313392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313123    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-envoy\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.313392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313157    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istiod-ca-cert\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.313392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313333    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-credential-socket" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:17.313392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313347    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-token\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-token\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.313729 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313399    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-kube-api-access-j46qk\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-kube-api-access-j46qk\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.313729 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313544    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istiod-ca-cert" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:17.313844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313787    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-podinfo\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.313899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313867    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-certs\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.313954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.313925    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-data\") pod \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\" (UID: \"411e28c0-bd7b-4294-b39f-0cb678be0fbf\") "
Oct 30 21:38:17.314338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.314177    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-certs" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:17.314577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.314534    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-data" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:17.314907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.314809    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.314907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.314838    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.314907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.314855    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.314907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.314871    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.314907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.314884    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.316272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.316241    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-envoy" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:17.316526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.316500    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-podinfo" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:17.316910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.316880    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-kube-api-access-j46qk" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "kube-api-access-j46qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:17.317001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.316927    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-token" pod "411e28c0-bd7b-4294-b39f-0cb678be0fbf" (UID: "411e28c0-bd7b-4294-b39f-0cb678be0fbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:17.415840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.415818    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.415840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.415841    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.415991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.415850    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j46qk\" (UniqueName: \"kubernetes.io/projected/411e28c0-bd7b-4294-b39f-0cb678be0fbf-kube-api-access-j46qk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.415991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.415861    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411e28c0-bd7b-4294-b39f-0cb678be0fbf-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.479253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.479227    5786 generic.go:356] "Generic (PLEG): container finished" podID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerID="f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1" exitCode=0
Oct 30 21:38:17.479364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.479264    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" event={"ID":"411e28c0-bd7b-4294-b39f-0cb678be0fbf","Type":"ContainerDied","Data":"f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1"}
Oct 30 21:38:17.479364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.479283    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"
Oct 30 21:38:17.479364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.479330    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p" event={"ID":"411e28c0-bd7b-4294-b39f-0cb678be0fbf","Type":"ContainerDied","Data":"f2be54dba858e55bf483227a9c30145493682c37dbd37f133ef96214751d0c7b"}
Oct 30 21:38:17.479364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.479359    5786 scope.go:122] "RemoveContainer" containerID="f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1"
Oct 30 21:38:17.480679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.480653    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:17.481711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.481692    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:38:17.491798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.491782    5786 scope.go:122] "RemoveContainer" containerID="fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151"
Oct 30 21:38:17.501744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.501721    5786 scope.go:122] "RemoveContainer" containerID="f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1"
Oct 30 21:38:17.502028 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:17.502002    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1\": container with ID starting with f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1 not found: ID does not exist" containerID="f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1"
Oct 30 21:38:17.512253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.502036    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1"} err="failed to get container status \"f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1\": rpc error: code = NotFound desc = could not find container \"f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1\": container with ID starting with f9e536832c54b46275d74bbc38c42f5ca34f326fef4499f018eb8f439973afb1 not found: ID does not exist"
Oct 30 21:38:17.512253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.502530    5786 scope.go:122] "RemoveContainer" containerID="fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151"
Oct 30 21:38:17.512253 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:17.502910    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151\": container with ID starting with fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151 not found: ID does not exist" containerID="fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151"
Oct 30 21:38:17.512253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.502946    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151"} err="failed to get container status \"fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151\": rpc error: code = NotFound desc = could not find container \"fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151\": container with ID starting with fa4bbecc346c4383abb7be5d6b2a779c4fbf8140063475dd2b63df8de50f2151 not found: ID does not exist"
Oct 30 21:38:17.869278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.855518    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"]
Oct 30 21:38:17.882234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:17.882056    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b25d536b-openshift-default-7974949447-tlf8p"]
Oct 30 21:38:18.282935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.282900    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"]
Oct 30 21:38:18.283736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.283715    5786 cpu_manager.go:414] "containerMap: removing container" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy"
Oct 30 21:38:18.283864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.283737    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy"
Oct 30 21:38:18.283864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.283760    5786 cpu_manager.go:414] "containerMap: removing container" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy"
Oct 30 21:38:18.283864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.283767    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy"
Oct 30 21:38:18.283864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.283807    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c7ae7dd9-4560-4a29-8425-0acc03ff57d5" containerName="maas-api"
Oct 30 21:38:18.283864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.283816    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7ae7dd9-4560-4a29-8425-0acc03ff57d5" containerName="maas-api"
Oct 30 21:38:18.284104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.284032    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7ae7dd9-4560-4a29-8425-0acc03ff57d5" containerName="maas-api"
Oct 30 21:38:18.284104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.284058    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy"
Oct 30 21:38:18.284104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.284068    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" containerName="istio-proxy"
Oct 30 21:38:18.287742 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.287722    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.343111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.343084    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:38:18.343525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.343317    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:38:18.344496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.344472    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"]
Oct 30 21:38:18.366033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.365998    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="411e28c0-bd7b-4294-b39f-0cb678be0fbf" path="/var/lib/kubelet/pods/411e28c0-bd7b-4294-b39f-0cb678be0fbf/volumes"
Oct 30 21:38:18.366737 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.366698    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7ae7dd9-4560-4a29-8425-0acc03ff57d5" path="/var/lib/kubelet/pods/c7ae7dd9-4560-4a29-8425-0acc03ff57d5/volumes"
Oct 30 21:38:18.433127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.433103    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmt6z\" (UniqueName: \"kubernetes.io/projected/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-kube-api-access-mmt6z\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.433275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.433250    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.433317 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.433292    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.534554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.534489    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmt6z\" (UniqueName: \"kubernetes.io/projected/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-kube-api-access-mmt6z\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.534678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.534586    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.534678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.534616    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.535197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.535176    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.537111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.537090    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.591759 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.591732    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmt6z\" (UniqueName: \"kubernetes.io/projected/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-kube-api-access-mmt6z\") pod \"maas-api-revoke-keys-e2e-ait-7379f500-bpd5t\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.600572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.600551    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:18.987381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:18.987357    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"]
Oct 30 21:38:18.987995 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:38:18.987972    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffae4807_f3bc_4ecb_950d_4c2807ef3e17.slice/crio-bf135009a9d85bc684916d6daeaca5d9b038486c0c0da7af9ae90d8c5fba44c2 WatchSource:0}: Error finding container bf135009a9d85bc684916d6daeaca5d9b038486c0c0da7af9ae90d8c5fba44c2: Status 404 returned error can't find the container with id bf135009a9d85bc684916d6daeaca5d9b038486c0c0da7af9ae90d8c5fba44c2
Oct 30 21:38:19.422443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:19.422411    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"]
Oct 30 21:38:19.422915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:19.422641    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" containerID="cri-o://53073a71031e400ace45d637f92a08b48e2555e0905c085806a0a2ce91588921" gracePeriod=30
Oct 30 21:38:19.500528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:19.500497    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t" event={"ID":"ffae4807-f3bc-4ecb-950d-4c2807ef3e17","Type":"ContainerStarted","Data":"74b031ca7917bbdccaf003074d0205bfdf3f8ec0e2a47e512b0e15e1c6665e1f"}
Oct 30 21:38:19.500680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:19.500553    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t" event={"ID":"ffae4807-f3bc-4ecb-950d-4c2807ef3e17","Type":"ContainerStarted","Data":"bf135009a9d85bc684916d6daeaca5d9b038486c0c0da7af9ae90d8c5fba44c2"}
Oct 30 21:38:20.513219 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:20.513168    5786 generic.go:356] "Generic (PLEG): container finished" podID="ffae4807-f3bc-4ecb-950d-4c2807ef3e17" containerID="74b031ca7917bbdccaf003074d0205bfdf3f8ec0e2a47e512b0e15e1c6665e1f" exitCode=0
Oct 30 21:38:20.513666 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:20.513237    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t" event={"ID":"ffae4807-f3bc-4ecb-950d-4c2807ef3e17","Type":"ContainerDied","Data":"74b031ca7917bbdccaf003074d0205bfdf3f8ec0e2a47e512b0e15e1c6665e1f"}
Oct 30 21:38:21.698922 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.698901    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:21.808886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.808826    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-kube-api-access-mmt6z\" (UniqueName: \"kubernetes.io/projected/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-kube-api-access-mmt6z\") pod \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") "
Oct 30 21:38:21.809007 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.808938    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-xks-ca\") pod \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") "
Oct 30 21:38:21.809074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.809045    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-openshift-ca\") pod \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\" (UID: \"ffae4807-f3bc-4ecb-950d-4c2807ef3e17\") "
Oct 30 21:38:21.809407 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.809384    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-openshift-ca" pod "ffae4807-f3bc-4ecb-950d-4c2807ef3e17" (UID: "ffae4807-f3bc-4ecb-950d-4c2807ef3e17"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:21.809727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.809710    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:21.810940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.810920    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-kube-api-access-mmt6z" pod "ffae4807-f3bc-4ecb-950d-4c2807ef3e17" (UID: "ffae4807-f3bc-4ecb-950d-4c2807ef3e17"). InnerVolumeSpecName "kube-api-access-mmt6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:21.811014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.810954    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-xks-ca" pod "ffae4807-f3bc-4ecb-950d-4c2807ef3e17" (UID: "ffae4807-f3bc-4ecb-950d-4c2807ef3e17"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:21.911731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.911703    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmt6z\" (UniqueName: \"kubernetes.io/projected/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-kube-api-access-mmt6z\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:21.911731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.911730    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ffae4807-f3bc-4ecb-950d-4c2807ef3e17-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:21.962162 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.962138    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"]
Oct 30 21:38:21.962370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:21.962346    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerName="maas-api" containerID="cri-o://c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df" gracePeriod=30
Oct 30 21:38:22.325942 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.325862    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:38:22.346921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.343751    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls\") pod \"667d7d7d-53bd-4597-8def-7c5025fc636c\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") "
Oct 30 21:38:22.346921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.343824    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-metrics-certs\") pod \"667d7d7d-53bd-4597-8def-7c5025fc636c\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") "
Oct 30 21:38:22.346921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.343970    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/667d7d7d-53bd-4597-8def-7c5025fc636c-kube-api-access-vgs6h\" (UniqueName: \"kubernetes.io/projected/667d7d7d-53bd-4597-8def-7c5025fc636c-kube-api-access-vgs6h\") pod \"667d7d7d-53bd-4597-8def-7c5025fc636c\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") "
Oct 30 21:38:22.346921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.344026    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667d7d7d-53bd-4597-8def-7c5025fc636c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667d7d7d-53bd-4597-8def-7c5025fc636c-openshift-service-ca\") pod \"667d7d7d-53bd-4597-8def-7c5025fc636c\" (UID: \"667d7d7d-53bd-4597-8def-7c5025fc636c\") "
Oct 30 21:38:22.346921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.345627    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667d7d7d-53bd-4597-8def-7c5025fc636c-openshift-service-ca" pod "667d7d7d-53bd-4597-8def-7c5025fc636c" (UID: "667d7d7d-53bd-4597-8def-7c5025fc636c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:22.346921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.346824    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-metrics-certs" pod "667d7d7d-53bd-4597-8def-7c5025fc636c" (UID: "667d7d7d-53bd-4597-8def-7c5025fc636c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:22.348105 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.348031    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls" pod "667d7d7d-53bd-4597-8def-7c5025fc636c" (UID: "667d7d7d-53bd-4597-8def-7c5025fc636c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:22.348761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.348735    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667d7d7d-53bd-4597-8def-7c5025fc636c-kube-api-access-vgs6h" pod "667d7d7d-53bd-4597-8def-7c5025fc636c" (UID: "667d7d7d-53bd-4597-8def-7c5025fc636c"). InnerVolumeSpecName "kube-api-access-vgs6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:22.371060 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:22.371026    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:38:22.446858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.446821    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:22.446858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.446854    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/667d7d7d-53bd-4597-8def-7c5025fc636c-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:22.447058 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.446868    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgs6h\" (UniqueName: \"kubernetes.io/projected/667d7d7d-53bd-4597-8def-7c5025fc636c-kube-api-access-vgs6h\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:22.447058 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.446882    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667d7d7d-53bd-4597-8def-7c5025fc636c-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:22.534856 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.534827    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"
Oct 30 21:38:22.535014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.534869    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t" event={"ID":"ffae4807-f3bc-4ecb-950d-4c2807ef3e17","Type":"ContainerDied","Data":"bf135009a9d85bc684916d6daeaca5d9b038486c0c0da7af9ae90d8c5fba44c2"}
Oct 30 21:38:22.535014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.534924    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf135009a9d85bc684916d6daeaca5d9b038486c0c0da7af9ae90d8c5fba44c2"
Oct 30 21:38:22.537242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.537132    5786 generic.go:356] "Generic (PLEG): container finished" podID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerID="c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df" exitCode=0
Oct 30 21:38:22.537544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.537362    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" event={"ID":"667d7d7d-53bd-4597-8def-7c5025fc636c","Type":"ContainerDied","Data":"c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df"}
Oct 30 21:38:22.537544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.537408    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v" event={"ID":"667d7d7d-53bd-4597-8def-7c5025fc636c","Type":"ContainerDied","Data":"97a01b776b0b291af936aa27d5345676afb75048e99d454c239d427d5f1923bf"}
Oct 30 21:38:22.537544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.537437    5786 scope.go:122] "RemoveContainer" containerID="c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df"
Oct 30 21:38:22.537738 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.537582    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"
Oct 30 21:38:22.550935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.550914    5786 scope.go:122] "RemoveContainer" containerID="b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec"
Oct 30 21:38:22.561196 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.561178    5786 scope.go:122] "RemoveContainer" containerID="c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df"
Oct 30 21:38:22.561509 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:22.561488    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df\": container with ID starting with c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df not found: ID does not exist" containerID="c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df"
Oct 30 21:38:22.561558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.561517    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df"} err="failed to get container status \"c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df\": rpc error: code = NotFound desc = could not find container \"c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df\": container with ID starting with c624ba15a21c814563c4710e88c9f2421d4264d0f1c992d07d8d5807eee8d1df not found: ID does not exist"
Oct 30 21:38:22.561558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.561533    5786 scope.go:122] "RemoveContainer" containerID="b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec"
Oct 30 21:38:22.561775 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:22.561759    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec\": container with ID starting with b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec not found: ID does not exist" containerID="b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec"
Oct 30 21:38:22.561815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.561781    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec"} err="failed to get container status \"b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec\": rpc error: code = NotFound desc = could not find container \"b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec\": container with ID starting with b4359a64436da094e7bcfa8a10b784735b476acb811c14f647bb4b8067418cec not found: ID does not exist"
Oct 30 21:38:22.741233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.740511    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"]
Oct 30 21:38:22.761997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:22.761961    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f787654-6755f8fdff-w2s6v"]
Oct 30 21:38:23.006174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.006084    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"]
Oct 30 21:38:23.007664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.007598    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ffae4807-f3bc-4ecb-950d-4c2807ef3e17" containerName="revoke-keys"
Oct 30 21:38:23.007864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.007846    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffae4807-f3bc-4ecb-950d-4c2807ef3e17" containerName="revoke-keys"
Oct 30 21:38:23.007988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.007973    5786 cpu_manager.go:414] "containerMap: removing container" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerName="maas-api"
Oct 30 21:38:23.008101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.008087    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerName="maas-api"
Oct 30 21:38:23.008196 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.008185    5786 cpu_manager.go:414] "containerMap: removing container" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerName="maas-api"
Oct 30 21:38:23.008301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.008289    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerName="maas-api"
Oct 30 21:38:23.008665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.008616    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffae4807-f3bc-4ecb-950d-4c2807ef3e17" containerName="revoke-keys"
Oct 30 21:38:23.008786 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.008774    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerName="maas-api"
Oct 30 21:38:23.008862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.008854    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" containerName="maas-api"
Oct 30 21:38:23.022187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.016185    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:23.080600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.078529    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-a7a6ff99\""
Oct 30 21:38:23.080600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.078713    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-a7a6ff99-dockercfg-mgkgc\""
Oct 30 21:38:23.094941 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.094893    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"]
Oct 30 21:38:23.126231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.123271    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50be3338-52f5-4401-bfe6-29cf30ddb1e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8\" (UID: \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:23.126231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.123520    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ht9l\" (UniqueName: \"kubernetes.io/projected/50be3338-52f5-4401-bfe6-29cf30ddb1e6-kube-api-access-7ht9l\") pod \"payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8\" (UID: \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:23.217389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.210574    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"]
Oct 30 21:38:23.217389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.216992    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:23.241794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.225258    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ht9l\" (UniqueName: \"kubernetes.io/projected/50be3338-52f5-4401-bfe6-29cf30ddb1e6-kube-api-access-7ht9l\") pod \"payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8\" (UID: \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:23.241794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.225392    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50be3338-52f5-4401-bfe6-29cf30ddb1e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8\" (UID: \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:23.241794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.225424    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz9sp\" (UniqueName: \"kubernetes.io/projected/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-kube-api-access-hz9sp\") pod \"payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx\" (UID: \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:23.241794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.225570    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx\" (UID: \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:23.241794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.226536    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50be3338-52f5-4401-bfe6-29cf30ddb1e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8\" (UID: \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:23.241794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.233565    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"]
Oct 30 21:38:23.252230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.246425    5786 patch_prober.go:29] interesting pod/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Oct 30 21:38:23.252230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.246490    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:38:23.363187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.355540    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx\" (UID: \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:23.363187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.355895    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz9sp\" (UniqueName: \"kubernetes.io/projected/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-kube-api-access-hz9sp\") pod \"payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx\" (UID: \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:23.417232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.416459    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx\" (UID: \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:23.512990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.512914    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz9sp\" (UniqueName: \"kubernetes.io/projected/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-kube-api-access-hz9sp\") pod \"payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx\" (UID: \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:23.542636 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.542598    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:23.548723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.544174    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ht9l\" (UniqueName: \"kubernetes.io/projected/50be3338-52f5-4401-bfe6-29cf30ddb1e6-kube-api-access-7ht9l\") pod \"payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8\" (UID: \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:23.701014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.700987    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:23.944947 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:38:23.944916    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8b5961_60a2_4040_8fc5_c0affa3d05f5.slice/crio-02d65aa8a606040f4f89f442f1848baba5921b5cdc5afc5b4972fbfc696490f1 WatchSource:0}: Error finding container 02d65aa8a606040f4f89f442f1848baba5921b5cdc5afc5b4972fbfc696490f1: Status 404 returned error can't find the container with id 02d65aa8a606040f4f89f442f1848baba5921b5cdc5afc5b4972fbfc696490f1
Oct 30 21:38:23.945422 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:23.945335    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"]
Oct 30 21:38:24.083002 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:38:24.082974    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50be3338_52f5_4401_bfe6_29cf30ddb1e6.slice/crio-e073195e29909f948ad08716e5d816702d19b06a9823be6015bbe1597ab0d316 WatchSource:0}: Error finding container e073195e29909f948ad08716e5d816702d19b06a9823be6015bbe1597ab0d316: Status 404 returned error can't find the container with id e073195e29909f948ad08716e5d816702d19b06a9823be6015bbe1597ab0d316
Oct 30 21:38:24.084702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.084679    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"]
Oct 30 21:38:24.372048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.372015    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="667d7d7d-53bd-4597-8def-7c5025fc636c" path="/var/lib/kubelet/pods/667d7d7d-53bd-4597-8def-7c5025fc636c/volumes"
Oct 30 21:38:24.647146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.646893    5786 generic.go:356] "Generic (PLEG): container finished" podID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerID="53073a71031e400ace45d637f92a08b48e2555e0905c085806a0a2ce91588921" exitCode=0
Oct 30 21:38:24.647146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.647005    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" event={"ID":"dcf64b51-cf8b-47ca-81e6-d9365f1de030","Type":"ContainerDied","Data":"53073a71031e400ace45d637f92a08b48e2555e0905c085806a0a2ce91588921"}
Oct 30 21:38:24.647146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.647057    5786 scope.go:122] "RemoveContainer" containerID="f9c905dac5f0dd5dd12dd21645e9b0d14acf80b7e528b2bfab47291ad8beec26"
Oct 30 21:38:24.672274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.671684    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8" event={"ID":"50be3338-52f5-4401-bfe6-29cf30ddb1e6","Type":"ContainerStarted","Data":"de68bedf1d08cdece8892b9a21d89361d1759c603b1118c2e3b98390a1c21b47"}
Oct 30 21:38:24.672274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.671743    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8" event={"ID":"50be3338-52f5-4401-bfe6-29cf30ddb1e6","Type":"ContainerStarted","Data":"e073195e29909f948ad08716e5d816702d19b06a9823be6015bbe1597ab0d316"}
Oct 30 21:38:24.673802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.673735    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:24.676948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.676926    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx" event={"ID":"9f8b5961-60a2-4040-8fc5-c0affa3d05f5","Type":"ContainerStarted","Data":"5a46a31a03769f733cd2ffc618af355c406e53ea2c1d67b571e08f23f93c76ea"}
Oct 30 21:38:24.677161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.677121    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx" event={"ID":"9f8b5961-60a2-4040-8fc5-c0affa3d05f5","Type":"ContainerStarted","Data":"02d65aa8a606040f4f89f442f1848baba5921b5cdc5afc5b4972fbfc696490f1"}
Oct 30 21:38:24.678366 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.678349    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:24.812757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:24.812694    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8" podStartSLOduration=2.812676115 podStartE2EDuration="2.812676115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:24.807613394 +0000 UTC m=+1560.678732514" watchObservedRunningTime="2026-10-30 21:38:24.812676115 +0000 UTC m=+1560.683795235"
Oct 30 21:38:25.046279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.037304    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx" podStartSLOduration=2.03728463 podStartE2EDuration="2.03728463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:25.030626071 +0000 UTC m=+1560.901745205" watchObservedRunningTime="2026-10-30 21:38:25.03728463 +0000 UTC m=+1560.908403749"
Oct 30 21:38:25.164291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.161983    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:38:25.253764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.253733    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-certs\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.253932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.253794    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-token\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-token\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.253932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.253814    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istiod-ca-cert\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.253932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.253844    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-data\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.253932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.253872    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-podinfo\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.254163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.253967    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-credential-socket\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.254163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.253996    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-socket\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.254163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.254015    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-certs" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:25.254163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.254075    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-kube-api-access-hvjzh\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-kube-api-access-hvjzh\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.254413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.254169    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-data" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:25.254413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.254318    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-socket" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:25.254413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.254345    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-credential-socket" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:25.254413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.254365    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-envoy\") pod \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\" (UID: \"dcf64b51-cf8b-47ca-81e6-d9365f1de030\") "
Oct 30 21:38:25.254413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.254402    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istiod-ca-cert" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:25.255048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.255030    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.255130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.255049    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.255130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.255059    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.255130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.255068    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.255130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.255076    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.256251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.256198    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-token" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:25.256348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.256267    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-podinfo" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:25.256844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.256827    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-envoy" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:25.256888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.256858    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-kube-api-access-hvjzh" pod "dcf64b51-cf8b-47ca-81e6-d9365f1de030" (UID: "dcf64b51-cf8b-47ca-81e6-d9365f1de030"). InnerVolumeSpecName "kube-api-access-hvjzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:25.356493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.356454    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.356493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.356485    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.356696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.356502    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcf64b51-cf8b-47ca-81e6-d9365f1de030-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.356696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.356515    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvjzh\" (UniqueName: \"kubernetes.io/projected/dcf64b51-cf8b-47ca-81e6-d9365f1de030-kube-api-access-hvjzh\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.685220 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.685140    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"
Oct 30 21:38:25.685220 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.685176    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm" event={"ID":"dcf64b51-cf8b-47ca-81e6-d9365f1de030","Type":"ContainerDied","Data":"8a706cf0dcd9b152cb5228c0f262ac5b7d7c05766f744c23b1ead4d7990f3671"}
Oct 30 21:38:25.685425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.685266    5786 scope.go:122] "RemoveContainer" containerID="53073a71031e400ace45d637f92a08b48e2555e0905c085806a0a2ce91588921"
Oct 30 21:38:25.882147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.882120    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"]
Oct 30 21:38:25.888544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:25.888523    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1d5266a5-openshift-default-c7d7cdb45-wsxkm"]
Oct 30 21:38:26.152451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.152368    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"]
Oct 30 21:38:26.154157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.154127    5786 cpu_manager.go:414] "containerMap: removing container" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy"
Oct 30 21:38:26.154157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.154157    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy"
Oct 30 21:38:26.154343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.154269    5786 cpu_manager.go:414] "containerMap: removing container" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy"
Oct 30 21:38:26.154343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.154280    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy"
Oct 30 21:38:26.154703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.154685    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy"
Oct 30 21:38:26.154782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.154724    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" containerName="istio-proxy"
Oct 30 21:38:26.161299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.160783    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.231543 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.231511    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"]
Oct 30 21:38:26.268512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268482    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.268690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268513    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.268690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268541    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.268800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268718    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.268800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268782    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.268904 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268822    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kw2x\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-kube-api-access-2kw2x\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.268904 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268855    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.268904 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268894    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.269036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.268948    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.366297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.366271    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcf64b51-cf8b-47ca-81e6-d9365f1de030" path="/var/lib/kubelet/pods/dcf64b51-cf8b-47ca-81e6-d9365f1de030/volumes"
Oct 30 21:38:26.369836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.369815    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.369933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.369844    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.369933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.369863    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.369983    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370037    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370066    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kw2x\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-kube-api-access-2kw2x\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370089    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370121    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370172    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370300    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370475 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370451    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370582    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370652    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.370709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.370674    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.372421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.372394    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.372807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.372789    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.426818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.426797    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.426921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.426909    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kw2x\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-kube-api-access-2kw2x\") pod \"maas-default-gateway-openshift-default-6cc458d887-qzlj5\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.474929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.474911    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:26.907687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.907654    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"]
Oct 30 21:38:26.907806 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:38:26.907709    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fdf533_e9a8_4410_910b_dfff5aaa73fb.slice/crio-f9f7b98999ef4e569b201d0d70456e6d5b40e40e30346975496d0ceb3217af6a WatchSource:0}: Error finding container f9f7b98999ef4e569b201d0d70456e6d5b40e40e30346975496d0ceb3217af6a: Status 404 returned error can't find the container with id f9f7b98999ef4e569b201d0d70456e6d5b40e40e30346975496d0ceb3217af6a
Oct 30 21:38:26.910634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.910603    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:26.910726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.910665    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:26.910726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:26.910695    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:27.705766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:27.705724    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" event={"ID":"03fdf533-e9a8-4410-910b-dfff5aaa73fb","Type":"ContainerStarted","Data":"34eec1c089350bd5e6d42e9d43af398ac9988be86d48b7af6424a175952d03a9"}
Oct 30 21:38:27.706240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:27.705784    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" event={"ID":"03fdf533-e9a8-4410-910b-dfff5aaa73fb","Type":"ContainerStarted","Data":"f9f7b98999ef4e569b201d0d70456e6d5b40e40e30346975496d0ceb3217af6a"}
Oct 30 21:38:27.937435 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:27.937388    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podStartSLOduration=1.937374447 podStartE2EDuration="1.937374447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:27.933231895 +0000 UTC m=+1563.804351004" watchObservedRunningTime="2026-10-30 21:38:27.937374447 +0000 UTC m=+1563.808493566"
Oct 30 21:38:28.478246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:28.476433    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:28.478246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:28.477991    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qzlj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:38:28.478246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:28.478043    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 21:38:29.476652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:29.476612    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qzlj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:38:29.477278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:29.476680    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 21:38:29.817878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:29.817782    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"]
Oct 30 21:38:29.920806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:29.920772    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"]
Oct 30 21:38:30.487233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:30.486339    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qzlj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:38:30.487233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:30.486411    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 21:38:31.479246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:31.476648    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qzlj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:38:31.479246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:31.476712    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 21:38:32.476562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:32.476533    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qzlj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:38:32.476970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:32.476596    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 21:38:33.475702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:33.475662    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qzlj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:38:33.475902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:33.475719    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 21:38:34.475255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:34.475190    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qzlj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:38:34.475633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:34.475282    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 21:38:34.818990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:34.818868    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9" podUID="8bc54c61-f740-45fc-81c8-a5c2412c06e7" containerName="payload-processing" containerID="cri-o://c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806" gracePeriod=25
Oct 30 21:38:34.921492 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:34.921452    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5" podUID="17677e40-4d28-49c1-8138-bd25a18e23f5" containerName="payload-pre-processing" containerID="cri-o://7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666" gracePeriod=25
Oct 30 21:38:35.134465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.133758    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:38:35.233832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.232849    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8bc54c61-f740-45fc-81c8-a5c2412c06e7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc54c61-f740-45fc-81c8-a5c2412c06e7-plugins-config-volume\") pod \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\" (UID: \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\") "
Oct 30 21:38:35.233832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.232899    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8bc54c61-f740-45fc-81c8-a5c2412c06e7-kube-api-access-p4mkq\" (UniqueName: \"kubernetes.io/projected/8bc54c61-f740-45fc-81c8-a5c2412c06e7-kube-api-access-p4mkq\") pod \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\" (UID: \"8bc54c61-f740-45fc-81c8-a5c2412c06e7\") "
Oct 30 21:38:35.233832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.233301    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc54c61-f740-45fc-81c8-a5c2412c06e7-plugins-config-volume" pod "8bc54c61-f740-45fc-81c8-a5c2412c06e7" (UID: "8bc54c61-f740-45fc-81c8-a5c2412c06e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:35.233832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.233783    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc54c61-f740-45fc-81c8-a5c2412c06e7-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:35.235947 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.235918    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bc54c61-f740-45fc-81c8-a5c2412c06e7-kube-api-access-p4mkq" pod "8bc54c61-f740-45fc-81c8-a5c2412c06e7" (UID: "8bc54c61-f740-45fc-81c8-a5c2412c06e7"). InnerVolumeSpecName "kube-api-access-p4mkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:35.260923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.260892    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:38:35.335870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.335837    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4mkq\" (UniqueName: \"kubernetes.io/projected/8bc54c61-f740-45fc-81c8-a5c2412c06e7-kube-api-access-p4mkq\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:35.437294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.437265    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17677e40-4d28-49c1-8138-bd25a18e23f5-kube-api-access-zqqnj\" (UniqueName: \"kubernetes.io/projected/17677e40-4d28-49c1-8138-bd25a18e23f5-kube-api-access-zqqnj\") pod \"17677e40-4d28-49c1-8138-bd25a18e23f5\" (UID: \"17677e40-4d28-49c1-8138-bd25a18e23f5\") "
Oct 30 21:38:35.437450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.437360    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17677e40-4d28-49c1-8138-bd25a18e23f5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17677e40-4d28-49c1-8138-bd25a18e23f5-plugins-config-volume\") pod \"17677e40-4d28-49c1-8138-bd25a18e23f5\" (UID: \"17677e40-4d28-49c1-8138-bd25a18e23f5\") "
Oct 30 21:38:35.437823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.437797    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17677e40-4d28-49c1-8138-bd25a18e23f5-plugins-config-volume" pod "17677e40-4d28-49c1-8138-bd25a18e23f5" (UID: "17677e40-4d28-49c1-8138-bd25a18e23f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:35.438566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.438544    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17677e40-4d28-49c1-8138-bd25a18e23f5-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:35.440401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.440380    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17677e40-4d28-49c1-8138-bd25a18e23f5-kube-api-access-zqqnj" pod "17677e40-4d28-49c1-8138-bd25a18e23f5" (UID: "17677e40-4d28-49c1-8138-bd25a18e23f5"). InnerVolumeSpecName "kube-api-access-zqqnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:35.540572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.540542    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqqnj\" (UniqueName: \"kubernetes.io/projected/17677e40-4d28-49c1-8138-bd25a18e23f5-kube-api-access-zqqnj\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:35.653281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.653220    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:35.653724 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.653703    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:35.654434 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.654418    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:38:35.812393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.812315    5786 generic.go:356] "Generic (PLEG): container finished" podID="17677e40-4d28-49c1-8138-bd25a18e23f5" containerID="7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666" exitCode=0
Oct 30 21:38:35.812393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.812358    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"
Oct 30 21:38:35.812606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.812400    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5" event={"ID":"17677e40-4d28-49c1-8138-bd25a18e23f5","Type":"ContainerDied","Data":"7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666"}
Oct 30 21:38:35.812606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.812454    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5" event={"ID":"17677e40-4d28-49c1-8138-bd25a18e23f5","Type":"ContainerDied","Data":"499c551378b06463f81ee1e9e72c1439367d02c4f21551463af5eb0bdbbc21ed"}
Oct 30 21:38:35.812606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.812481    5786 scope.go:122] "RemoveContainer" containerID="7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666"
Oct 30 21:38:35.814472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.814450    5786 generic.go:356] "Generic (PLEG): container finished" podID="8bc54c61-f740-45fc-81c8-a5c2412c06e7" containerID="c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806" exitCode=0
Oct 30 21:38:35.814593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.814494    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"
Oct 30 21:38:35.814593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.814530    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9" event={"ID":"8bc54c61-f740-45fc-81c8-a5c2412c06e7","Type":"ContainerDied","Data":"c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806"}
Oct 30 21:38:35.814593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.814578    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9" event={"ID":"8bc54c61-f740-45fc-81c8-a5c2412c06e7","Type":"ContainerDied","Data":"9aebb7fc03d30f646a65406231ef36757cd7809979c6552fec87f8f5d4b7d743"}
Oct 30 21:38:35.831648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.826470    5786 scope.go:122] "RemoveContainer" containerID="7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666"
Oct 30 21:38:35.831648 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:35.826762    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666\": container with ID starting with 7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666 not found: ID does not exist" containerID="7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666"
Oct 30 21:38:35.831648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.826786    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666"} err="failed to get container status \"7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666\": rpc error: code = NotFound desc = could not find container \"7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666\": container with ID starting with 7535fbf159386b01be1a9d13ebc3dfd8df2e3d03b89008a3262821368e70e666 not found: ID does not exist"
Oct 30 21:38:35.831648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.826804    5786 scope.go:122] "RemoveContainer" containerID="c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806"
Oct 30 21:38:35.849414 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.849397    5786 scope.go:122] "RemoveContainer" containerID="c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806"
Oct 30 21:38:35.849745 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:35.849720    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806\": container with ID starting with c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806 not found: ID does not exist" containerID="c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806"
Oct 30 21:38:35.849851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:35.849750    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806"} err="failed to get container status \"c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806\": rpc error: code = NotFound desc = could not find container \"c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806\": container with ID starting with c486bd5b6af444098a8bd4098328ee7d8320f8844aae24e502002b2fd3280806 not found: ID does not exist"
Oct 30 21:38:36.020910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:36.020879    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"]
Oct 30 21:38:36.025625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:36.025593    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7379f500-7d5768d9ff-s9fh5"]
Oct 30 21:38:36.130591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:36.130506    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"]
Oct 30 21:38:36.135092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:36.135068    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7379f500-644568b6d9-k45x9"]
Oct 30 21:38:36.360868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:36.360839    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17677e40-4d28-49c1-8138-bd25a18e23f5" path="/var/lib/kubelet/pods/17677e40-4d28-49c1-8138-bd25a18e23f5/volumes"
Oct 30 21:38:36.361305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:36.361290    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8bc54c61-f740-45fc-81c8-a5c2412c06e7" path="/var/lib/kubelet/pods/8bc54c61-f740-45fc-81c8-a5c2412c06e7/volumes"
Oct 30 21:38:36.697468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:36.697443    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:38:36.697914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:36.697504    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:38:37.055842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:37.055758    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"]
Oct 30 21:38:37.060528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:37.060504    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e269cbb1-tt8jk"]
Oct 30 21:38:38.361535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:38.361502    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad0dc3d6-c86f-4d37-8196-f887411d0683" path="/var/lib/kubelet/pods/ad0dc3d6-c86f-4d37-8196-f887411d0683/volumes"
Oct 30 21:38:41.231813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.231781    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"]
Oct 30 21:38:41.232511 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.232496    5786 cpu_manager.go:414] "containerMap: removing container" podUID="17677e40-4d28-49c1-8138-bd25a18e23f5" containerName="payload-pre-processing"
Oct 30 21:38:41.232566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.232512    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17677e40-4d28-49c1-8138-bd25a18e23f5" containerName="payload-pre-processing"
Oct 30 21:38:41.232566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.232535    5786 cpu_manager.go:414] "containerMap: removing container" podUID="8bc54c61-f740-45fc-81c8-a5c2412c06e7" containerName="payload-processing"
Oct 30 21:38:41.232566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.232541    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8bc54c61-f740-45fc-81c8-a5c2412c06e7" containerName="payload-processing"
Oct 30 21:38:41.232739 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.232727    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="8bc54c61-f740-45fc-81c8-a5c2412c06e7" containerName="payload-processing"
Oct 30 21:38:41.232851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.232754    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="17677e40-4d28-49c1-8138-bd25a18e23f5" containerName="payload-pre-processing"
Oct 30 21:38:41.236447 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.236427    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.328908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.328878    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.329185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.329159    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.329334 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.329271    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.329445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.329426    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.329512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.329497    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.329927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.329559    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kd9f\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-kube-api-access-6kd9f\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.329927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.329596    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.329927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.329643    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.329927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.329706    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.345441 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.345413    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"]
Oct 30 21:38:41.430524 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.430484    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.430700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.430676    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.430767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.430753    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kd9f\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-kube-api-access-6kd9f\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.430834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.430814    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.430892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.430879    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.430955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.430943    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.431127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.431103    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.431287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.431144    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.431287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.431185    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.431458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.431436    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.431570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.431516    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.431570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.431564    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.431681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.431634    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.431948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.431923    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.433117 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.433102    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.433341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.433324    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.486115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.486067    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.486196 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.486135    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kd9f\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-kube-api-access-6kd9f\") pod \"maas-default-gateway-openshift-default-6cc458d887-qs95c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.550030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.549940    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:41.987506 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:38:41.987479    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84691101_48fe_4c5f_ae64_f6faa4bbae0c.slice/crio-56c3efea7486b71f34a4c73516d83f1c9a74759b49c3a3219873a1e3bc2088f6 WatchSource:0}: Error finding container 56c3efea7486b71f34a4c73516d83f1c9a74759b49c3a3219873a1e3bc2088f6: Status 404 returned error can't find the container with id 56c3efea7486b71f34a4c73516d83f1c9a74759b49c3a3219873a1e3bc2088f6
Oct 30 21:38:41.987841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.987818    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"]
Oct 30 21:38:41.990076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.990049    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:41.990135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.990109    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:41.990170 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:41.990137    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:38:42.924235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:42.920695    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" event={"ID":"84691101-48fe-4c5f-ae64-f6faa4bbae0c","Type":"ContainerStarted","Data":"26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee"}
Oct 30 21:38:42.924235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:42.920759    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" event={"ID":"84691101-48fe-4c5f-ae64-f6faa4bbae0c","Type":"ContainerStarted","Data":"56c3efea7486b71f34a4c73516d83f1c9a74759b49c3a3219873a1e3bc2088f6"}
Oct 30 21:38:43.069381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:43.055031    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podStartSLOduration=2.055002257 podStartE2EDuration="2.055002257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:43.03888226 +0000 UTC m=+1578.910001379" watchObservedRunningTime="2026-10-30 21:38:43.055002257 +0000 UTC m=+1578.926121398"
Oct 30 21:38:43.550678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:43.550647    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:43.563330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:43.563296    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qs95c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 21:38:43.563570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:43.563542    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 21:38:44.572232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:44.551328    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qs95c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 21:38:44.572232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:44.551384    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 21:38:44.822363 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:44.822294    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"]
Oct 30 21:38:44.882396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:44.882363    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7379f500-bpd5t"]
Oct 30 21:38:45.419284 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:45.419253    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"]
Oct 30 21:38:45.419500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:45.419457    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy" containerID="cri-o://f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65" gracePeriod=30
Oct 30 21:38:45.561180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:45.561144    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qs95c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 21:38:45.561507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:45.561476    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 21:38:46.383770 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:46.383741    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffae4807-f3bc-4ecb-950d-4c2807ef3e17" path="/var/lib/kubelet/pods/ffae4807-f3bc-4ecb-950d-4c2807ef3e17/volumes"
Oct 30 21:38:46.550811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:46.550783    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qs95c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 21:38:46.550974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:46.550836    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 21:38:47.550751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:47.550718    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qs95c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 21:38:47.551115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:47.550768    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 21:38:48.551255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:48.551186    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qs95c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 21:38:48.551653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:48.551284    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 21:38:49.550861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:49.550825    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qs95c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 21:38:49.551029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:49.550890    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 21:38:50.675093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.674972    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:50.676352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.676284    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:50.677116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.677078    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:38:50.761526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.761502    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:38:50.853307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853278    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-token\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-token\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853315    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c65c2976-ed99-4599-93cd-c1fa38b5df34-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65c2976-ed99-4599-93cd-c1fa38b5df34-istiod-ca-cert\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853375    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-certs\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853396    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-podinfo\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853417    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-data\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853528    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-envoy\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853583    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-kube-api-access-54khr\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-kube-api-access-54khr\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853630    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-credential-socket\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853673    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-socket\") pod \"c65c2976-ed99-4599-93cd-c1fa38b5df34\" (UID: \"c65c2976-ed99-4599-93cd-c1fa38b5df34\") "
Oct 30 21:38:50.853929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853670    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-certs" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:50.854152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853784    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-data" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:50.854152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853829    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-credential-socket" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:50.854152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853892    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65c2976-ed99-4599-93cd-c1fa38b5df34-istiod-ca-cert" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:50.854152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.853969    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-socket" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:50.854429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.854402    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:50.854429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.854426    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65c2976-ed99-4599-93cd-c1fa38b5df34-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:50.854536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.854435    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:50.854536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.854444    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:50.854536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.854452    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:50.855768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.855744    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-token" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:50.855960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.855945    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-envoy" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:50.856282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.856261    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-podinfo" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:50.856282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.856268    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-kube-api-access-54khr" pod "c65c2976-ed99-4599-93cd-c1fa38b5df34" (UID: "c65c2976-ed99-4599-93cd-c1fa38b5df34"). InnerVolumeSpecName "kube-api-access-54khr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:50.955661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.955637    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:50.955661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.955659    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:50.955792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.955669    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65c2976-ed99-4599-93cd-c1fa38b5df34-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:50.955792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:50.955678    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54khr\" (UniqueName: \"kubernetes.io/projected/c65c2976-ed99-4599-93cd-c1fa38b5df34-kube-api-access-54khr\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:51.055035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.054938    5786 generic.go:356] "Generic (PLEG): container finished" podID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerID="f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65" exitCode=0
Oct 30 21:38:51.055035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.055011    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"
Oct 30 21:38:51.055405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.055372    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" event={"ID":"c65c2976-ed99-4599-93cd-c1fa38b5df34","Type":"ContainerDied","Data":"f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65"}
Oct 30 21:38:51.055515 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.055456    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6" event={"ID":"c65c2976-ed99-4599-93cd-c1fa38b5df34","Type":"ContainerDied","Data":"557cf1a1833436969d4df03914903f741bc33a00f2997b39f4bde507a7f034e8"}
Oct 30 21:38:51.055515 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.055489    5786 scope.go:122] "RemoveContainer" containerID="f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65"
Oct 30 21:38:51.070979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.070963    5786 scope.go:122] "RemoveContainer" containerID="c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f"
Oct 30 21:38:51.080750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.080729    5786 scope.go:122] "RemoveContainer" containerID="f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65"
Oct 30 21:38:51.081044 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:51.081027    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65\": container with ID starting with f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65 not found: ID does not exist" containerID="f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65"
Oct 30 21:38:51.081106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.081051    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65"} err="failed to get container status \"f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65\": rpc error: code = NotFound desc = could not find container \"f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65\": container with ID starting with f64b3c855c585288ba7a254319112fe80834ec88362b0c2f474f628387fe9b65 not found: ID does not exist"
Oct 30 21:38:51.081106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.081067    5786 scope.go:122] "RemoveContainer" containerID="c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f"
Oct 30 21:38:51.081367 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:51.081351    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f\": container with ID starting with c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f not found: ID does not exist" containerID="c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f"
Oct 30 21:38:51.081451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.081372    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f"} err="failed to get container status \"c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f\": rpc error: code = NotFound desc = could not find container \"c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f\": container with ID starting with c2a696a9282e4eacd10b17a2d740f327385c0658447f6a4297eb334a7e08dc7f not found: ID does not exist"
Oct 30 21:38:51.249178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.249144    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"]
Oct 30 21:38:51.253820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:51.253795    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7379f500-openshift-default-774b84bcf5-g4hq6"]
Oct 30 21:38:52.355479 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:52.355451    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:38:52.361291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:52.361264    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" path="/var/lib/kubelet/pods/c65c2976-ed99-4599-93cd-c1fa38b5df34/volumes"
Oct 30 21:38:54.053120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:54.053045    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"]
Oct 30 21:38:54.102326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:54.102297    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ace1c9-lbjk7"]
Oct 30 21:38:54.361619 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:54.361543    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c8cc2d7-5a53-453a-97be-e361e7da133d" path="/var/lib/kubelet/pods/0c8cc2d7-5a53-453a-97be-e361e7da133d/volumes"
Oct 30 21:38:56.051938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:56.051908    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"]
Oct 30 21:38:56.101059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:56.101029    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf9e350d-2kssw"]
Oct 30 21:38:56.360975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:56.360905    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f5aff79-0f83-4de4-91f1-4410180655c4" path="/var/lib/kubelet/pods/8f5aff79-0f83-4de4-91f1-4410180655c4/volumes"
Oct 30 21:38:58.952356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:58.952326    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"]
Oct 30 21:38:58.953116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:58.953097    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy"
Oct 30 21:38:58.953165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:58.953118    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy"
Oct 30 21:38:58.953165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:58.953133    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy"
Oct 30 21:38:58.953165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:58.953138    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy"
Oct 30 21:38:58.953297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:58.953291    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy"
Oct 30 21:38:58.953327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:58.953300    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c65c2976-ed99-4599-93cd-c1fa38b5df34" containerName="istio-proxy"
Oct 30 21:38:58.958789 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:58.958769    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.015557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.015538    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"]
Oct 30 21:38:59.059507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.059487    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-a7a6ff99-serving-cert\""
Oct 30 21:38:59.062300 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.062271    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2fcb22-4078-4029-bf55-3076896689ef-openshift-service-ca\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.062376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.062322    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5drgv\" (UniqueName: \"kubernetes.io/projected/4b2fcb22-4078-4029-bf55-3076896689ef-kube-api-access-5drgv\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.062567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.062549    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.062673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.062657    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-metrics-certs\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.163909 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.163872    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.164076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.163970    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-metrics-certs\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.164076 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:59.164000    5786 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-serving-cert: secret "maas-api-e2e-migrate-a7a6ff99-serving-cert" not found
Oct 30 21:38:59.164076 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:38:59.164049    5786 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls podName:4b2fcb22-4078-4029-bf55-3076896689ef nodeName:}" failed. No retries permitted until 2026-10-30 21:38:59.664035359 +0000 UTC m=+1595.535154487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls") pod "maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp" (UID: "4b2fcb22-4078-4029-bf55-3076896689ef") : secret "maas-api-e2e-migrate-a7a6ff99-serving-cert" not found
Oct 30 21:38:59.164226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.164130    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2fcb22-4078-4029-bf55-3076896689ef-openshift-service-ca\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.164226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.164169    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5drgv\" (UniqueName: \"kubernetes.io/projected/4b2fcb22-4078-4029-bf55-3076896689ef-kube-api-access-5drgv\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.164751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.164732    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2fcb22-4078-4029-bf55-3076896689ef-openshift-service-ca\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.166496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.166479    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-metrics-certs\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.235568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.235390    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5drgv\" (UniqueName: \"kubernetes.io/projected/4b2fcb22-4078-4029-bf55-3076896689ef-kube-api-access-5drgv\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.691920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.691890    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.694351 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.694329    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls\") pod \"maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:38:59.873844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:38:59.873812    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:39:00.271791 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:39:00.271757    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2fcb22_4078_4029_bf55_3076896689ef.slice/crio-9dbfe4d9bc43c9331be1c0bab354fb0b65e40cb80341ea40b78b12a676299a0d WatchSource:0}: Error finding container 9dbfe4d9bc43c9331be1c0bab354fb0b65e40cb80341ea40b78b12a676299a0d: Status 404 returned error can't find the container with id 9dbfe4d9bc43c9331be1c0bab354fb0b65e40cb80341ea40b78b12a676299a0d
Oct 30 21:39:00.272389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:00.272349    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"]
Oct 30 21:39:01.140262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:01.140236    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp" event={"ID":"4b2fcb22-4078-4029-bf55-3076896689ef","Type":"ContainerStarted","Data":"9dbfe4d9bc43c9331be1c0bab354fb0b65e40cb80341ea40b78b12a676299a0d"}
Oct 30 21:39:02.139328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:02.139274    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"]
Oct 30 21:39:02.167844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:02.167799    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp" event={"ID":"4b2fcb22-4078-4029-bf55-3076896689ef","Type":"ContainerStarted","Data":"990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07"}
Oct 30 21:39:02.168102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:02.168081    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:39:02.172133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:02.172108    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0c6a5c7-w24hw"]
Oct 30 21:39:02.285655 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:02.285607    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp" podStartSLOduration=3.432365075 podStartE2EDuration="4.285593546s" totalImagesPullingTime="853.228471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:02.28268222 +0000 UTC m=+1598.153801340" watchObservedRunningTime="2026-10-30 21:39:02.285593546 +0000 UTC m=+1598.156712679"
Oct 30 21:39:02.361336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:02.361313    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="818ee224-eaab-49a5-8fe6-bcb2196fe626" path="/var/lib/kubelet/pods/818ee224-eaab-49a5-8fe6-bcb2196fe626/volumes"
Oct 30 21:39:08.182380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:08.182353    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:39:11.052828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:11.052796    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"]
Oct 30 21:39:11.101915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:11.101887    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7bd362e-4dp5g"]
Oct 30 21:39:12.360794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:12.360759    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b16ec9e7-0734-438f-95df-5734fa44d405" path="/var/lib/kubelet/pods/b16ec9e7-0734-438f-95df-5734fa44d405/volumes"
Oct 30 21:39:19.050755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:19.050724    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"]
Oct 30 21:39:19.100838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:19.100807    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9322e825-dqrfm"]
Oct 30 21:39:20.360366 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:20.360337    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19baf13a-39c2-430b-9ae4-c46a2d483a4c" path="/var/lib/kubelet/pods/19baf13a-39c2-430b-9ae4-c46a2d483a4c/volumes"
Oct 30 21:39:22.355245 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:39:22.355185    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:39:25.087231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:25.076733    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"]
Oct 30 21:39:25.111532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:25.111499    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"]
Oct 30 21:39:25.114240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:25.114218    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8732eae-7qlnc"]
Oct 30 21:39:25.118186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:25.118168    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c026a502-h4sds"]
Oct 30 21:39:25.436049 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:25.436022    5786 scope.go:122] "RemoveContainer" containerID="69ee8ca91e3a52f2a69cf90867cac80a764018fe36c17378482dc3ccdccb48a8"
Oct 30 21:39:25.446513 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:25.446498    5786 scope.go:122] "RemoveContainer" containerID="3f0d76b1fd0f2c8cc79f1de766ebbde810b778afbbeedf93adfe4b086856eb62"
Oct 30 21:39:25.455802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:25.455786    5786 scope.go:122] "RemoveContainer" containerID="30cfd947c8357fafd694db842cc30ef1d4dcbb662142b6fb013078bfdec50852"
Oct 30 21:39:25.465145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:25.465126    5786 scope.go:122] "RemoveContainer" containerID="63693c7db2f89e971e5f5be0a5e70dfa610303c11986aabec86e272f51827bbd"
Oct 30 21:39:26.389936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:26.389898    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5347e77d-10d8-49f4-b60f-9e8250af963d" path="/var/lib/kubelet/pods/5347e77d-10d8-49f4-b60f-9e8250af963d/volumes"
Oct 30 21:39:26.391357 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:26.391334    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8520185-8f72-472c-bc54-a8dfb09da2db" path="/var/lib/kubelet/pods/b8520185-8f72-472c-bc54-a8dfb09da2db/volumes"
Oct 30 21:39:28.482031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.481991    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"]
Oct 30 21:39:28.489200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.489180    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.527829 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.527802    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"]
Oct 30 21:39:28.576512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.576482    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:39:28.576720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.576652    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:39:28.650581 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.650549    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.650795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.650776    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-xks-ca\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.650854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.650841    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4ltj\" (UniqueName: \"kubernetes.io/projected/54108272-5457-4321-8ecb-2a34adad8a16-kube-api-access-f4ltj\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.756758 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.756667    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.777229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.757193    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-xks-ca\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.777447 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.777421    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4ltj\" (UniqueName: \"kubernetes.io/projected/54108272-5457-4321-8ecb-2a34adad8a16-kube-api-access-f4ltj\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.777578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.759961    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.778257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.778196    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-xks-ca\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:28.837278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:28.837250    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4ltj\" (UniqueName: \"kubernetes.io/projected/54108272-5457-4321-8ecb-2a34adad8a16-kube-api-access-f4ltj\") pod \"maas-api-revoke-sub-520ea9d9-tsk26\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:29.102915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:29.102813    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:29.511994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:29.511966    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"]
Oct 30 21:39:29.512536 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:39:29.512512    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54108272_5457_4321_8ecb_2a34adad8a16.slice/crio-724d2ba92960dffb3355c7d83de6511a64d2df10376696f590d0e3366cda047c WatchSource:0}: Error finding container 724d2ba92960dffb3355c7d83de6511a64d2df10376696f590d0e3366cda047c: Status 404 returned error can't find the container with id 724d2ba92960dffb3355c7d83de6511a64d2df10376696f590d0e3366cda047c
Oct 30 21:39:30.424567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:30.424534    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26" event={"ID":"54108272-5457-4321-8ecb-2a34adad8a16","Type":"ContainerStarted","Data":"5565e5f262aad34e1a25ef55655b85483ebe369dbb4e7d8599f557395ad56588"}
Oct 30 21:39:30.424751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:30.424590    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26" event={"ID":"54108272-5457-4321-8ecb-2a34adad8a16","Type":"ContainerStarted","Data":"724d2ba92960dffb3355c7d83de6511a64d2df10376696f590d0e3366cda047c"}
Oct 30 21:39:30.540868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:30.540822    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26" podStartSLOduration=2.540807704 podStartE2EDuration="2.540807704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:30.538269387 +0000 UTC m=+1626.409388507" watchObservedRunningTime="2026-10-30 21:39:30.540807704 +0000 UTC m=+1626.411926817"
Oct 30 21:39:30.954986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:30.954954    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"]
Oct 30 21:39:30.961316 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:30.961281    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.057253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.040507    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bf1b02c7-openshift-default-dockercfg-gm6xd\""
Oct 30 21:39:31.228591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.228505    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"]
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.275513    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed744068-8b59-4580-b1ec-e34a5f40a63e-istiod-ca-cert\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.275575    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-podinfo\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.275616    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-data\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.275639    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jdpz\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-kube-api-access-4jdpz\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.275710    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-credential-socket\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.275743    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-socket\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.275776    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-token\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.276006    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-certs\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.280261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.276059    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-envoy\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.417487    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-podinfo\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.417574    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-data\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.417611    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jdpz\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-kube-api-access-4jdpz\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.417709    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-credential-socket\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.417741    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-socket\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.417775    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-token\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.417954    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-certs\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.418015    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-envoy\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.418303    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed744068-8b59-4580-b1ec-e34a5f40a63e-istiod-ca-cert\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.421227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.419126    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed744068-8b59-4580-b1ec-e34a5f40a63e-istiod-ca-cert\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.430439 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.430395    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-socket\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.430699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.430678    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-data\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.445027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.444973    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-credential-socket\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.445442 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.445410    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-certs\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.455297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.451951    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-envoy\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.505907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.505829    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jdpz\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-kube-api-access-4jdpz\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.527234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.523734    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-token\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.528605 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.528581    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-podinfo\") pod \"e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") " pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:31.530765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.530736    5786 generic.go:356] "Generic (PLEG): container finished" podID="54108272-5457-4321-8ecb-2a34adad8a16" containerID="5565e5f262aad34e1a25ef55655b85483ebe369dbb4e7d8599f557395ad56588" exitCode=0
Oct 30 21:39:31.531017 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.530995    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26" event={"ID":"54108272-5457-4321-8ecb-2a34adad8a16","Type":"ContainerDied","Data":"5565e5f262aad34e1a25ef55655b85483ebe369dbb4e7d8599f557395ad56588"}
Oct 30 21:39:31.662421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:31.662393    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:32.220298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.220272    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"]
Oct 30 21:39:32.220861 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:39:32.220837    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded744068_8b59_4580_b1ec_e34a5f40a63e.slice/crio-316dbdc32fe1b83598ba3cb2b8e078cc4f06559d06b135dc6b2793b8e57de062 WatchSource:0}: Error finding container 316dbdc32fe1b83598ba3cb2b8e078cc4f06559d06b135dc6b2793b8e57de062: Status 404 returned error can't find the container with id 316dbdc32fe1b83598ba3cb2b8e078cc4f06559d06b135dc6b2793b8e57de062
Oct 30 21:39:32.223385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.223355    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:39:32.223495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.223426    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:39:32.223495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.223469    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:39:32.543679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.543540    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" event={"ID":"ed744068-8b59-4580-b1ec-e34a5f40a63e","Type":"ContainerStarted","Data":"4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761"}
Oct 30 21:39:32.543679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.543643    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" event={"ID":"ed744068-8b59-4580-b1ec-e34a5f40a63e","Type":"ContainerStarted","Data":"316dbdc32fe1b83598ba3cb2b8e078cc4f06559d06b135dc6b2793b8e57de062"}
Oct 30 21:39:32.667253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.665643    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" podStartSLOduration=2.6655991759999997 podStartE2EDuration="2.665599176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:32.662026656 +0000 UTC m=+1628.533145795" watchObservedRunningTime="2026-10-30 21:39:32.665599176 +0000 UTC m=+1628.536718297"
Oct 30 21:39:32.667732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.667329    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:32.720270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.720243    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:32.801283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.792910    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54108272-5457-4321-8ecb-2a34adad8a16-kube-api-access-f4ltj\" (UniqueName: \"kubernetes.io/projected/54108272-5457-4321-8ecb-2a34adad8a16-kube-api-access-f4ltj\") pod \"54108272-5457-4321-8ecb-2a34adad8a16\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") "
Oct 30 21:39:32.801283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.793112    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-xks-ca\") pod \"54108272-5457-4321-8ecb-2a34adad8a16\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") "
Oct 30 21:39:32.801283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.793314    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-openshift-ca\") pod \"54108272-5457-4321-8ecb-2a34adad8a16\" (UID: \"54108272-5457-4321-8ecb-2a34adad8a16\") "
Oct 30 21:39:32.813231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.811464    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-openshift-ca" pod "54108272-5457-4321-8ecb-2a34adad8a16" (UID: "54108272-5457-4321-8ecb-2a34adad8a16"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:39:32.855795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.854892    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-xks-ca" pod "54108272-5457-4321-8ecb-2a34adad8a16" (UID: "54108272-5457-4321-8ecb-2a34adad8a16"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:39:32.867526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.867488    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54108272-5457-4321-8ecb-2a34adad8a16-kube-api-access-f4ltj" pod "54108272-5457-4321-8ecb-2a34adad8a16" (UID: "54108272-5457-4321-8ecb-2a34adad8a16"). InnerVolumeSpecName "kube-api-access-f4ltj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:39:32.923500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.914911    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4ltj\" (UniqueName: \"kubernetes.io/projected/54108272-5457-4321-8ecb-2a34adad8a16-kube-api-access-f4ltj\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:39:32.923500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.914949    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:39:32.923500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:32.914969    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54108272-5457-4321-8ecb-2a34adad8a16-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:39:33.523365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:33.523334    5786 patch_prober.go:29] interesting pod/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:39:33.523562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:33.523389    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:39:33.554507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:33.554475    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"
Oct 30 21:39:33.554679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:33.554518    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26" event={"ID":"54108272-5457-4321-8ecb-2a34adad8a16","Type":"ContainerDied","Data":"724d2ba92960dffb3355c7d83de6511a64d2df10376696f590d0e3366cda047c"}
Oct 30 21:39:33.554679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:33.554579    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="724d2ba92960dffb3355c7d83de6511a64d2df10376696f590d0e3366cda047c"
Oct 30 21:39:33.668152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:33.668119    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:34.564429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:34.564403    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:34.565284 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:34.565264    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:39:35.573421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:35.573360    5786 generic.go:356] "Generic (PLEG): container finished" podID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerID="ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e" exitCode=0
Oct 30 21:39:35.573781 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:35.573426    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" event={"ID":"b39fbcd0-dd33-4041-8496-c9310c19856e","Type":"ContainerDied","Data":"ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e"}
Oct 30 21:39:35.573822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:35.573790    5786 scope.go:122] "RemoveContainer" containerID="ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e"
Oct 30 21:39:35.575076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:35.575051    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:39:35.575131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:35.575115    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:39:35.575166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:35.575146    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:39:36.585996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:36.585942    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" event={"ID":"b39fbcd0-dd33-4041-8496-c9310c19856e","Type":"ContainerStarted","Data":"511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc"}
Oct 30 21:39:36.599404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:36.599379    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:39:36.599643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:36.599618    5786 patch_prober.go:29] interesting pod/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:39:36.599712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:36.599686    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 21:39:37.599935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:37.599898    5786 patch_prober.go:29] interesting pod/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:39:37.600446 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:37.599954    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 21:39:38.599626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:38.599590    5786 patch_prober.go:29] interesting pod/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:39:38.599832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:38.599657    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 21:39:39.599909 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:39.599879    5786 patch_prober.go:29] interesting pod/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:39:39.600338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:39.599929    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 21:39:40.600325 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:40.600296    5786 patch_prober.go:29] interesting pod/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:39:40.600777 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:40.600352    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 21:39:41.603374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:41.603345    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:39:41.603848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:41.603831    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:39:41.604485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:41.604465    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:39:43.054120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:43.054090    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"]
Oct 30 21:39:43.102715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:43.102691    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36404357-jp2tn"]
Oct 30 21:39:44.051871 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:44.051838    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"]
Oct 30 21:39:44.101276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:44.101251    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c28d39c0-g8bgm"]
Oct 30 21:39:44.361659 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:44.361580    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629" path="/var/lib/kubelet/pods/ca1e3db9-2a03-4bb3-a2bb-8fbc694ae629/volumes"
Oct 30 21:39:44.362166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:44.362151    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f997344c-1340-4b42-a44e-1e3493d22100" path="/var/lib/kubelet/pods/f997344c-1340-4b42-a44e-1e3493d22100/volumes"
Oct 30 21:39:47.065162 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:47.065131    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"]
Oct 30 21:39:47.110514 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:47.110487    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8f87036-6x7tm"]
Oct 30 21:39:48.361351 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:48.361316    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91be25e6-e157-4623-98c7-a3da3eaef22f" path="/var/lib/kubelet/pods/91be25e6-e157-4623-98c7-a3da3eaef22f/volumes"
Oct 30 21:39:49.061035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:49.061004    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"]
Oct 30 21:39:49.109459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:49.109431    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c9dc34f-4gnxg"]
Oct 30 21:39:50.321444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.321413    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"]
Oct 30 21:39:50.322140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.322126    5786 cpu_manager.go:414] "containerMap: removing container" podUID="54108272-5457-4321-8ecb-2a34adad8a16" containerName="revoke-keys"
Oct 30 21:39:50.322183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.322141    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54108272-5457-4321-8ecb-2a34adad8a16" containerName="revoke-keys"
Oct 30 21:39:50.322282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.322273    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="54108272-5457-4321-8ecb-2a34adad8a16" containerName="revoke-keys"
Oct 30 21:39:50.325365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.325351    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.361821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.361796    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74dac01f-a3e2-41f9-a7d1-7acde185faca" path="/var/lib/kubelet/pods/74dac01f-a3e2-41f9-a7d1-7acde185faca/volumes"
Oct 30 21:39:50.411109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.411076    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.411287 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.411132    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lw4z\" (UniqueName: \"kubernetes.io/projected/cf42f84e-4045-458a-bfd8-1a2e67f9b294-kube-api-access-8lw4z\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.411465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.411439    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.426624 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.426601    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:39:50.426864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.426846    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:39:50.434045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.434024    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"]
Oct 30 21:39:50.512820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.512794    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.513048 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.513026    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.513138 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.513099    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lw4z\" (UniqueName: \"kubernetes.io/projected/cf42f84e-4045-458a-bfd8-1a2e67f9b294-kube-api-access-8lw4z\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.513572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.513546    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.515860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.515842    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.569451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.569426    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lw4z\" (UniqueName: \"kubernetes.io/projected/cf42f84e-4045-458a-bfd8-1a2e67f9b294-kube-api-access-8lw4z\") pod \"maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:50.638406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:50.638347    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:51.026078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:51.026051    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"]
Oct 30 21:39:51.026446 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:39:51.026422    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf42f84e_4045_458a_bfd8_1a2e67f9b294.slice/crio-2b5ccb47f1dbbee94e7f9200494168c00aeae16123fd014332872ebe5c0bf961 WatchSource:0}: Error finding container 2b5ccb47f1dbbee94e7f9200494168c00aeae16123fd014332872ebe5c0bf961: Status 404 returned error can't find the container with id 2b5ccb47f1dbbee94e7f9200494168c00aeae16123fd014332872ebe5c0bf961
Oct 30 21:39:51.052102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:51.052080    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"]
Oct 30 21:39:51.074796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:51.074775    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5953ef0b-wlwls"]
Oct 30 21:39:51.714552 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:51.714519    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q" event={"ID":"cf42f84e-4045-458a-bfd8-1a2e67f9b294","Type":"ContainerStarted","Data":"c85f0ca8291338e10e380eea65fbde8e71e2e8dee733419a3cc0d788547acf87"}
Oct 30 21:39:51.715009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:51.714577    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q" event={"ID":"cf42f84e-4045-458a-bfd8-1a2e67f9b294","Type":"ContainerStarted","Data":"2b5ccb47f1dbbee94e7f9200494168c00aeae16123fd014332872ebe5c0bf961"}
Oct 30 21:39:51.831774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:51.831725    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q" podStartSLOduration=1.831711203 podStartE2EDuration="1.831711203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:51.828416191 +0000 UTC m=+1647.699535310" watchObservedRunningTime="2026-10-30 21:39:51.831711203 +0000 UTC m=+1647.702830391"
Oct 30 21:39:52.357810 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:39:52.357776    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:39:52.362374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:52.362348    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29b66767-b1f7-45ca-a5d4-88be10b6aeea" path="/var/lib/kubelet/pods/29b66767-b1f7-45ca-a5d4-88be10b6aeea/volumes"
Oct 30 21:39:52.723580 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:52.723555    5786 generic.go:356] "Generic (PLEG): container finished" podID="cf42f84e-4045-458a-bfd8-1a2e67f9b294" containerID="c85f0ca8291338e10e380eea65fbde8e71e2e8dee733419a3cc0d788547acf87" exitCode=0
Oct 30 21:39:52.723966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:52.723638    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q" event={"ID":"cf42f84e-4045-458a-bfd8-1a2e67f9b294","Type":"ContainerDied","Data":"c85f0ca8291338e10e380eea65fbde8e71e2e8dee733419a3cc0d788547acf87"}
Oct 30 21:39:53.057136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:53.057043    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"]
Oct 30 21:39:53.061419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:53.061393    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7cc2a2ae-6g4m2"]
Oct 30 21:39:53.887750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:53.887729    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:54.062321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.062249    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf42f84e-4045-458a-bfd8-1a2e67f9b294-kube-api-access-8lw4z\" (UniqueName: \"kubernetes.io/projected/cf42f84e-4045-458a-bfd8-1a2e67f9b294-kube-api-access-8lw4z\") pod \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") "
Oct 30 21:39:54.062468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.062382    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-openshift-ca\") pod \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") "
Oct 30 21:39:54.062594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.062580    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-xks-ca\") pod \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\" (UID: \"cf42f84e-4045-458a-bfd8-1a2e67f9b294\") "
Oct 30 21:39:54.062709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.062688    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-openshift-ca" pod "cf42f84e-4045-458a-bfd8-1a2e67f9b294" (UID: "cf42f84e-4045-458a-bfd8-1a2e67f9b294"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:39:54.063065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.063047    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:39:54.064528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.064503    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf42f84e-4045-458a-bfd8-1a2e67f9b294-kube-api-access-8lw4z" pod "cf42f84e-4045-458a-bfd8-1a2e67f9b294" (UID: "cf42f84e-4045-458a-bfd8-1a2e67f9b294"). InnerVolumeSpecName "kube-api-access-8lw4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:39:54.064870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.064846    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-xks-ca" pod "cf42f84e-4045-458a-bfd8-1a2e67f9b294" (UID: "cf42f84e-4045-458a-bfd8-1a2e67f9b294"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:39:54.164542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.164517    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf42f84e-4045-458a-bfd8-1a2e67f9b294-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:39:54.164542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.164542    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lw4z\" (UniqueName: \"kubernetes.io/projected/cf42f84e-4045-458a-bfd8-1a2e67f9b294-kube-api-access-8lw4z\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:39:54.360842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.360779    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3b3704b-94f6-4fae-bbb4-a0de88973316" path="/var/lib/kubelet/pods/d3b3704b-94f6-4fae-bbb4-a0de88973316/volumes"
Oct 30 21:39:54.741946 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.741922    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"
Oct 30 21:39:54.742111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.741963    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q" event={"ID":"cf42f84e-4045-458a-bfd8-1a2e67f9b294","Type":"ContainerDied","Data":"2b5ccb47f1dbbee94e7f9200494168c00aeae16123fd014332872ebe5c0bf961"}
Oct 30 21:39:54.742111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:54.742009    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b5ccb47f1dbbee94e7f9200494168c00aeae16123fd014332872ebe5c0bf961"
Oct 30 21:39:58.054089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:58.054058    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"]
Oct 30 21:39:58.103234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:58.103199    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1b584c07-5qbjg"]
Oct 30 21:39:58.361775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:58.361697    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c" path="/var/lib/kubelet/pods/5c8e4a48-dcc7-40fd-9df4-8db8723c6b5c/volumes"
Oct 30 21:39:59.718229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.713326    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"]
Oct 30 21:39:59.718229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.714672    5786 cpu_manager.go:414] "containerMap: removing container" podUID="cf42f84e-4045-458a-bfd8-1a2e67f9b294" containerName="revoke-keys"
Oct 30 21:39:59.718229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.714695    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf42f84e-4045-458a-bfd8-1a2e67f9b294" containerName="revoke-keys"
Oct 30 21:39:59.718229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.714875    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf42f84e-4045-458a-bfd8-1a2e67f9b294" containerName="revoke-keys"
Oct 30 21:39:59.731640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.731610    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:39:59.778385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.778355    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq\" (UID: \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:39:59.779061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.779035    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw885\" (UniqueName: \"kubernetes.io/projected/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-kube-api-access-dw885\") pod \"payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq\" (UID: \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:39:59.796678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.796634    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bf1b02c7\""
Oct 30 21:39:59.797753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.797729    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bf1b02c7-dockercfg-tl6rn\""
Oct 30 21:39:59.825245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.824378    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"]
Oct 30 21:39:59.882724 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.882674    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq\" (UID: \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:39:59.883132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.883108    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw885\" (UniqueName: \"kubernetes.io/projected/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-kube-api-access-dw885\") pod \"payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq\" (UID: \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:39:59.884649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.884626    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq\" (UID: \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:39:59.907166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.907121    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"]
Oct 30 21:40:00.003700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:39:59.997522    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw885\" (UniqueName: \"kubernetes.io/projected/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-kube-api-access-dw885\") pod \"payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq\" (UID: \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:40:00.014111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.014079    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:00.023273 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.023240    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"]
Oct 30 21:40:00.208875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.208477    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:40:00.231394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.231364    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6bb0a20-dd91-4a81-9d6e-e8db064517af-plugins-config-volume\") pod \"payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq\" (UID: \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\") " pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:00.239279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.231914    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2t76\" (UniqueName: \"kubernetes.io/projected/b6bb0a20-dd91-4a81-9d6e-e8db064517af-kube-api-access-m2t76\") pod \"payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq\" (UID: \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\") " pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:00.341330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.341301    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2t76\" (UniqueName: \"kubernetes.io/projected/b6bb0a20-dd91-4a81-9d6e-e8db064517af-kube-api-access-m2t76\") pod \"payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq\" (UID: \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\") " pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:00.341485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.341393    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6bb0a20-dd91-4a81-9d6e-e8db064517af-plugins-config-volume\") pod \"payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq\" (UID: \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\") " pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:00.342160 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.342142    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6bb0a20-dd91-4a81-9d6e-e8db064517af-plugins-config-volume\") pod \"payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq\" (UID: \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\") " pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:00.406562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.406530    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2t76\" (UniqueName: \"kubernetes.io/projected/b6bb0a20-dd91-4a81-9d6e-e8db064517af-kube-api-access-m2t76\") pod \"payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq\" (UID: \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\") " pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:00.601006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.600935    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:00.689097 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:00.689067    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07686cd0_e97e_43bf_9dc6_f73d0f3a1ee3.slice/crio-6c893381731a676005c5073d60a02070a7e617a1063a4496e028a466840fbdf3 WatchSource:0}: Error finding container 6c893381731a676005c5073d60a02070a7e617a1063a4496e028a466840fbdf3: Status 404 returned error can't find the container with id 6c893381731a676005c5073d60a02070a7e617a1063a4496e028a466840fbdf3
Oct 30 21:40:00.689258 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.689188    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"]
Oct 30 21:40:00.946837 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.946742    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq" event={"ID":"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3","Type":"ContainerStarted","Data":"93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967"}
Oct 30 21:40:00.946837 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.946802    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq" event={"ID":"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3","Type":"ContainerStarted","Data":"6c893381731a676005c5073d60a02070a7e617a1063a4496e028a466840fbdf3"}
Oct 30 21:40:00.947387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.946917    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:40:00.979455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:00.979418    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"]
Oct 30 21:40:00.980304 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:00.980283    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6bb0a20_dd91_4a81_9d6e_e8db064517af.slice/crio-efe65411bb6d6d03237b72795a77b63a0ea8084c63a2dc54b61ae5c0cb480681 WatchSource:0}: Error finding container efe65411bb6d6d03237b72795a77b63a0ea8084c63a2dc54b61ae5c0cb480681: Status 404 returned error can't find the container with id efe65411bb6d6d03237b72795a77b63a0ea8084c63a2dc54b61ae5c0cb480681
Oct 30 21:40:01.063457 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:01.063409    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq" podStartSLOduration=2.063395597 podStartE2EDuration="2.063395597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:01.060986532 +0000 UTC m=+1656.932105650" watchObservedRunningTime="2026-10-30 21:40:01.063395597 +0000 UTC m=+1656.934514716"
Oct 30 21:40:02.016339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:02.016180    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq" event={"ID":"b6bb0a20-dd91-4a81-9d6e-e8db064517af","Type":"ContainerStarted","Data":"97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3"}
Oct 30 21:40:02.016339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:02.016252    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq" event={"ID":"b6bb0a20-dd91-4a81-9d6e-e8db064517af","Type":"ContainerStarted","Data":"efe65411bb6d6d03237b72795a77b63a0ea8084c63a2dc54b61ae5c0cb480681"}
Oct 30 21:40:02.016339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:02.016301    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:02.132948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:02.132898    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq" podStartSLOduration=3.132883611 podStartE2EDuration="3.132883611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:02.129009997 +0000 UTC m=+1658.000129118" watchObservedRunningTime="2026-10-30 21:40:02.132883611 +0000 UTC m=+1658.004002730"
Oct 30 21:40:02.828660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:02.828630    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"]
Oct 30 21:40:02.951540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:02.951508    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"]
Oct 30 21:40:04.052057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:04.052030    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"]
Oct 30 21:40:04.056314 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:04.056287    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0ade4058-77qxh"]
Oct 30 21:40:04.743268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:04.713979    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f7fae30-d7b9-459b-9040-675b18644d99" path="/var/lib/kubelet/pods/0f7fae30-d7b9-459b-9040-675b18644d99/volumes"
Oct 30 21:40:05.055928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:05.055840    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"]
Oct 30 21:40:05.101553 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:05.101525    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3f62a96-vtxl6"]
Oct 30 21:40:06.367363 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:06.367334    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="670254e0-d1c2-44ec-af01-6ab9897f26fb" path="/var/lib/kubelet/pods/670254e0-d1c2-44ec-af01-6ab9897f26fb/volumes"
Oct 30 21:40:07.829241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:07.829163    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx" podUID="9f8b5961-60a2-4040-8fc5-c0affa3d05f5" containerName="payload-processing" containerID="cri-o://5a46a31a03769f733cd2ffc618af355c406e53ea2c1d67b571e08f23f93c76ea" gracePeriod=25
Oct 30 21:40:07.951974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:07.951917    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8" podUID="50be3338-52f5-4401-bfe6-29cf30ddb1e6" containerName="payload-pre-processing" containerID="cri-o://de68bedf1d08cdece8892b9a21d89361d1759c603b1118c2e3b98390a1c21b47" gracePeriod=25
Oct 30 21:40:08.098559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.098358    5786 generic.go:356] "Generic (PLEG): container finished" podID="50be3338-52f5-4401-bfe6-29cf30ddb1e6" containerID="de68bedf1d08cdece8892b9a21d89361d1759c603b1118c2e3b98390a1c21b47" exitCode=0
Oct 30 21:40:08.098559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.098478    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8" event={"ID":"50be3338-52f5-4401-bfe6-29cf30ddb1e6","Type":"ContainerDied","Data":"de68bedf1d08cdece8892b9a21d89361d1759c603b1118c2e3b98390a1c21b47"}
Oct 30 21:40:08.100916 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.100765    5786 generic.go:356] "Generic (PLEG): container finished" podID="9f8b5961-60a2-4040-8fc5-c0affa3d05f5" containerID="5a46a31a03769f733cd2ffc618af355c406e53ea2c1d67b571e08f23f93c76ea" exitCode=0
Oct 30 21:40:08.100916 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.100799    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx" event={"ID":"9f8b5961-60a2-4040-8fc5-c0affa3d05f5","Type":"ContainerDied","Data":"5a46a31a03769f733cd2ffc618af355c406e53ea2c1d67b571e08f23f93c76ea"}
Oct 30 21:40:08.100916 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.100838    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx" event={"ID":"9f8b5961-60a2-4040-8fc5-c0affa3d05f5","Type":"ContainerDied","Data":"02d65aa8a606040f4f89f442f1848baba5921b5cdc5afc5b4972fbfc696490f1"}
Oct 30 21:40:08.100916 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.100870    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02d65aa8a606040f4f89f442f1848baba5921b5cdc5afc5b4972fbfc696490f1"
Oct 30 21:40:08.113946 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.113863    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:40:08.176676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.176013    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-plugins-config-volume\") pod \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\" (UID: \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\") "
Oct 30 21:40:08.176676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.176405    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-kube-api-access-hz9sp\" (UniqueName: \"kubernetes.io/projected/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-kube-api-access-hz9sp\") pod \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\" (UID: \"9f8b5961-60a2-4040-8fc5-c0affa3d05f5\") "
Oct 30 21:40:08.183828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.183799    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-plugins-config-volume" pod "9f8b5961-60a2-4040-8fc5-c0affa3d05f5" (UID: "9f8b5961-60a2-4040-8fc5-c0affa3d05f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:08.185583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.185552    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-kube-api-access-hz9sp" pod "9f8b5961-60a2-4040-8fc5-c0affa3d05f5" (UID: "9f8b5961-60a2-4040-8fc5-c0affa3d05f5"). InnerVolumeSpecName "kube-api-access-hz9sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:08.244560 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.244539    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:40:08.280111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.280084    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz9sp\" (UniqueName: \"kubernetes.io/projected/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-kube-api-access-hz9sp\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:08.280257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.280112    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8b5961-60a2-4040-8fc5-c0affa3d05f5-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:08.380761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.380732    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50be3338-52f5-4401-bfe6-29cf30ddb1e6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50be3338-52f5-4401-bfe6-29cf30ddb1e6-plugins-config-volume\") pod \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\" (UID: \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\") "
Oct 30 21:40:08.380911 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.380798    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50be3338-52f5-4401-bfe6-29cf30ddb1e6-kube-api-access-7ht9l\" (UniqueName: \"kubernetes.io/projected/50be3338-52f5-4401-bfe6-29cf30ddb1e6-kube-api-access-7ht9l\") pod \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\" (UID: \"50be3338-52f5-4401-bfe6-29cf30ddb1e6\") "
Oct 30 21:40:08.381137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.381115    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50be3338-52f5-4401-bfe6-29cf30ddb1e6-plugins-config-volume" pod "50be3338-52f5-4401-bfe6-29cf30ddb1e6" (UID: "50be3338-52f5-4401-bfe6-29cf30ddb1e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:08.381671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.381595    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50be3338-52f5-4401-bfe6-29cf30ddb1e6-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:08.383613 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.383586    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50be3338-52f5-4401-bfe6-29cf30ddb1e6-kube-api-access-7ht9l" pod "50be3338-52f5-4401-bfe6-29cf30ddb1e6" (UID: "50be3338-52f5-4401-bfe6-29cf30ddb1e6"). InnerVolumeSpecName "kube-api-access-7ht9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:08.483356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:08.483334    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ht9l\" (UniqueName: \"kubernetes.io/projected/50be3338-52f5-4401-bfe6-29cf30ddb1e6-kube-api-access-7ht9l\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:09.057925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.057894    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"]
Oct 30 21:40:09.107068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.106986    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d44bc68-fgvd9"]
Oct 30 21:40:09.111125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.111106    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"
Oct 30 21:40:09.111257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.111139    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8" event={"ID":"50be3338-52f5-4401-bfe6-29cf30ddb1e6","Type":"ContainerDied","Data":"e073195e29909f948ad08716e5d816702d19b06a9823be6015bbe1597ab0d316"}
Oct 30 21:40:09.111257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.111201    5786 scope.go:122] "RemoveContainer" containerID="de68bedf1d08cdece8892b9a21d89361d1759c603b1118c2e3b98390a1c21b47"
Oct 30 21:40:09.111352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.111328    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"
Oct 30 21:40:09.328892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.328866    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"]
Oct 30 21:40:09.332854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.332836    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7a6ff99-5dff48db99-tdzdx"]
Oct 30 21:40:09.439478 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.439453    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"]
Oct 30 21:40:09.445644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:09.445622    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7a6ff99-799d48f64c-9xlk8"]
Oct 30 21:40:10.361356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:10.361313    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50be3338-52f5-4401-bfe6-29cf30ddb1e6" path="/var/lib/kubelet/pods/50be3338-52f5-4401-bfe6-29cf30ddb1e6/volumes"
Oct 30 21:40:10.361823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:10.361808    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f8b5961-60a2-4040-8fc5-c0affa3d05f5" path="/var/lib/kubelet/pods/9f8b5961-60a2-4040-8fc5-c0affa3d05f5/volumes"
Oct 30 21:40:10.362332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:10.362315    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc32932b-a1ea-4c89-8f51-a171fd161203" path="/var/lib/kubelet/pods/fc32932b-a1ea-4c89-8f51-a171fd161203/volumes"
Oct 30 21:40:12.018166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:12.018140    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:40:14.033355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:14.033328    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:40:18.054063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:18.054030    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"]
Oct 30 21:40:18.103090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:18.103059    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9be34f0b-kkmtm"]
Oct 30 21:40:18.361141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:18.361069    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c42756e3-4b4c-4227-9440-1903037bc148" path="/var/lib/kubelet/pods/c42756e3-4b4c-4227-9440-1903037bc148/volumes"
Oct 30 21:40:21.710380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:21.710337    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qzlj5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:40:21.710889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:21.710396    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:40:22.053242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:22.053131    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"]
Oct 30 21:40:22.101783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:22.101749    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843f6340-fwgzk"]
Oct 30 21:40:22.356449 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:40:22.356368    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:40:22.361062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:22.361034    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="864a9c89-2cc3-4865-b8c9-49aa3a95564c" path="/var/lib/kubelet/pods/864a9c89-2cc3-4865-b8c9-49aa3a95564c/volumes"
Oct 30 21:40:24.244354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.244306    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"]
Oct 30 21:40:24.246042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.246013    5786 cpu_manager.go:414] "containerMap: removing container" podUID="50be3338-52f5-4401-bfe6-29cf30ddb1e6" containerName="payload-pre-processing"
Oct 30 21:40:24.246226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.246193    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50be3338-52f5-4401-bfe6-29cf30ddb1e6" containerName="payload-pre-processing"
Oct 30 21:40:24.246362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.246349    5786 cpu_manager.go:414] "containerMap: removing container" podUID="9f8b5961-60a2-4040-8fc5-c0affa3d05f5" containerName="payload-processing"
Oct 30 21:40:24.246450 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.246440    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f8b5961-60a2-4040-8fc5-c0affa3d05f5" containerName="payload-processing"
Oct 30 21:40:24.246788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.246773    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="50be3338-52f5-4401-bfe6-29cf30ddb1e6" containerName="payload-pre-processing"
Oct 30 21:40:24.246889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.246879    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f8b5961-60a2-4040-8fc5-c0affa3d05f5" containerName="payload-processing"
Oct 30 21:40:24.262242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.259602    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.314595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.314560    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-be8b73-openshift-default-dockercfg-rbc5n\""
Oct 30 21:40:24.421237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.394031    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-envoy\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.421237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.394096    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-token\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.421237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.394150    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-podinfo\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.421237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.394246    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-socket\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.421237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.394316    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klr4r\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-kube-api-access-klr4r\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.421237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.394342    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bebe1c-95dc-4834-b836-bbd28f21f95d-istiod-ca-cert\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.423490 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.423455    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-certs\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.467563 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.467526    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-data\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.467851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.467829    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-credential-socket\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.486557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.486531    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"]
Oct 30 21:40:24.690242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.689642    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-certs\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.690906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.690886    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-data\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.691095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.691081    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-credential-socket\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.691507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.691486    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-envoy\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.691745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.691720    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-token\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.691988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.691968    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-podinfo\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.709219 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.703310    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-certs\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.711424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.711358    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-socket\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.711630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.711613    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-socket\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.712574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.712551    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klr4r\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-kube-api-access-klr4r\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.713479 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.713413    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bebe1c-95dc-4834-b836-bbd28f21f95d-istiod-ca-cert\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.714318 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.712771    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bebe1c-95dc-4834-b836-bbd28f21f95d-istiod-ca-cert\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.715384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.715338    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-credential-socket\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.715708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.715690    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-data\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.755063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.755025    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-podinfo\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.755278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.755249    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-envoy\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.763200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.763135    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-token\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:24.818234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:24.816007    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klr4r\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-kube-api-access-klr4r\") pod \"e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") " pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:25.015495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.015420    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:25.274297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.274228    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"]
Oct 30 21:40:25.274812 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.274445    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp" podUID="4b2fcb22-4078-4029-bf55-3076896689ef" containerName="maas-api" containerID="cri-o://990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07" gracePeriod=30
Oct 30 21:40:25.451676 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:25.451026    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69bebe1c_95dc_4834_b836_bbd28f21f95d.slice/crio-0c7a7f2f568fa669e2643c25b183e821635dd5ac1342b82fec32a442dfa51cea WatchSource:0}: Error finding container 0c7a7f2f568fa669e2643c25b183e821635dd5ac1342b82fec32a442dfa51cea: Status 404 returned error can't find the container with id 0c7a7f2f568fa669e2643c25b183e821635dd5ac1342b82fec32a442dfa51cea
Oct 30 21:40:25.451676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.451489    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"]
Oct 30 21:40:25.454780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.454748    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:40:25.454928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.454827    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:40:25.454928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.454857    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:40:25.614863 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.614681    5786 scope.go:122] "RemoveContainer" containerID="fa0471cc13ce873ae9d4c85f16bd99e748f503f0128c9d3169e397517af59abe"
Oct 30 21:40:25.804877 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.794004    5786 scope.go:122] "RemoveContainer" containerID="8d9334116f9ee9a3cbcd21e846b881f186aff7faa4c89c591a5569992e7b3774"
Oct 30 21:40:25.916501 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.914566    5786 scope.go:122] "RemoveContainer" containerID="1f7b512a124f07e4ca8026c04c63130ca94393d2d96e4095960945bf39d24888"
Oct 30 21:40:25.931096 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.931062    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:40:25.942919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.942900    5786 scope.go:122] "RemoveContainer" containerID="5c3221203f1c5f02c4d1b876ad4575975f104ff6a9b9f894c6764a151c8bea96"
Oct 30 21:40:25.956696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.956678    5786 scope.go:122] "RemoveContainer" containerID="9f01e366d2e47e44590a12ca4c4e65747c753c20d1fb3db3b15c1e589dd4184e"
Oct 30 21:40:25.973927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.973907    5786 scope.go:122] "RemoveContainer" containerID="61d2f01407d55d6b7655cc68163948bc52971430ad89ba71c10bc161cde22840"
Oct 30 21:40:25.986064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.986040    5786 scope.go:122] "RemoveContainer" containerID="9a011a5c9bbbabc1be4328bbf03d1ad3f999a52c96158dac71bc82100d03f01b"
Oct 30 21:40:25.998268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:25.998250    5786 scope.go:122] "RemoveContainer" containerID="8165bab7250c804f7a045281e71bb243e8d3953d3470fdfdc9cac9b70664d6e9"
Oct 30 21:40:26.037676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.037650    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-metrics-certs\") pod \"4b2fcb22-4078-4029-bf55-3076896689ef\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") "
Oct 30 21:40:26.037807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.037787    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b2fcb22-4078-4029-bf55-3076896689ef-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2fcb22-4078-4029-bf55-3076896689ef-openshift-service-ca\") pod \"4b2fcb22-4078-4029-bf55-3076896689ef\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") "
Oct 30 21:40:26.037899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.037877    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls\") pod \"4b2fcb22-4078-4029-bf55-3076896689ef\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") "
Oct 30 21:40:26.038065 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.038021    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b2fcb22-4078-4029-bf55-3076896689ef-kube-api-access-5drgv\" (UniqueName: \"kubernetes.io/projected/4b2fcb22-4078-4029-bf55-3076896689ef-kube-api-access-5drgv\") pod \"4b2fcb22-4078-4029-bf55-3076896689ef\" (UID: \"4b2fcb22-4078-4029-bf55-3076896689ef\") "
Oct 30 21:40:26.038325 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.038301    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b2fcb22-4078-4029-bf55-3076896689ef-openshift-service-ca" pod "4b2fcb22-4078-4029-bf55-3076896689ef" (UID: "4b2fcb22-4078-4029-bf55-3076896689ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:26.038826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.038809    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2fcb22-4078-4029-bf55-3076896689ef-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:26.040513 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.040492    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-metrics-certs" pod "4b2fcb22-4078-4029-bf55-3076896689ef" (UID: "4b2fcb22-4078-4029-bf55-3076896689ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:40:26.041165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.041132    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls" pod "4b2fcb22-4078-4029-bf55-3076896689ef" (UID: "4b2fcb22-4078-4029-bf55-3076896689ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:40:26.041353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.041336    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2fcb22-4078-4029-bf55-3076896689ef-kube-api-access-5drgv" pod "4b2fcb22-4078-4029-bf55-3076896689ef" (UID: "4b2fcb22-4078-4029-bf55-3076896689ef"). InnerVolumeSpecName "kube-api-access-5drgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:26.140832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.140808    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:26.140832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.140832    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2fcb22-4078-4029-bf55-3076896689ef-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:26.140986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.140844    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5drgv\" (UniqueName: \"kubernetes.io/projected/4b2fcb22-4078-4029-bf55-3076896689ef-kube-api-access-5drgv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:26.392571 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.392541    5786 generic.go:356] "Generic (PLEG): container finished" podID="4b2fcb22-4078-4029-bf55-3076896689ef" containerID="990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07" exitCode=0
Oct 30 21:40:26.392964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.392574    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"
Oct 30 21:40:26.392964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.392581    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp" event={"ID":"4b2fcb22-4078-4029-bf55-3076896689ef","Type":"ContainerDied","Data":"990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07"}
Oct 30 21:40:26.392964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.392627    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp" event={"ID":"4b2fcb22-4078-4029-bf55-3076896689ef","Type":"ContainerDied","Data":"9dbfe4d9bc43c9331be1c0bab354fb0b65e40cb80341ea40b78b12a676299a0d"}
Oct 30 21:40:26.392964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.392660    5786 scope.go:122] "RemoveContainer" containerID="990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07"
Oct 30 21:40:26.394616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.394586    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" event={"ID":"69bebe1c-95dc-4834-b836-bbd28f21f95d","Type":"ContainerStarted","Data":"5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23"}
Oct 30 21:40:26.394727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.394647    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" event={"ID":"69bebe1c-95dc-4834-b836-bbd28f21f95d","Type":"ContainerStarted","Data":"0c7a7f2f568fa669e2643c25b183e821635dd5ac1342b82fec32a442dfa51cea"}
Oct 30 21:40:26.403710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.403689    5786 scope.go:122] "RemoveContainer" containerID="990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07"
Oct 30 21:40:26.403987 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:40:26.403971    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07\": container with ID starting with 990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07 not found: ID does not exist" containerID="990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07"
Oct 30 21:40:26.404044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.403997    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07"} err="failed to get container status \"990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07\": rpc error: code = NotFound desc = could not find container \"990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07\": container with ID starting with 990b414f258d5e5f3ae2122a472d7f96c34e74349a5ae3513a5ab5b038a3cf07 not found: ID does not exist"
Oct 30 21:40:26.564456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.564410    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" podStartSLOduration=2.564395883 podStartE2EDuration="2.564395883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:26.562322628 +0000 UTC m=+1682.433441750" watchObservedRunningTime="2026-10-30 21:40:26.564395883 +0000 UTC m=+1682.435515072"
Oct 30 21:40:26.680600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.675041    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"]
Oct 30 21:40:26.737236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:26.731796    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a7a6ff99-68675685c9-vw2jp"]
Oct 30 21:40:27.015755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:27.015684    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:27.021866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:27.021842    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:27.404034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:27.403940    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"]
Oct 30 21:40:27.406463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:27.406439    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:27.407606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:27.407586    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:40:27.408456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:27.408434    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a7a6ff99-7sq7q"]
Oct 30 21:40:27.512489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:27.512457    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"]
Oct 30 21:40:27.512923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:27.512888    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy" containerID="cri-o://511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc" gracePeriod=30
Oct 30 21:40:28.361332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:28.361299    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b2fcb22-4078-4029-bf55-3076896689ef" path="/var/lib/kubelet/pods/4b2fcb22-4078-4029-bf55-3076896689ef/volumes"
Oct 30 21:40:28.361867 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:28.361852    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf42f84e-4045-458a-bfd8-1a2e67f9b294" path="/var/lib/kubelet/pods/cf42f84e-4045-458a-bfd8-1a2e67f9b294/volumes"
Oct 30 21:40:33.071965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.071938    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"]
Oct 30 21:40:33.072704 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.072690    5786 cpu_manager.go:414] "containerMap: removing container" podUID="4b2fcb22-4078-4029-bf55-3076896689ef" containerName="maas-api"
Oct 30 21:40:33.072757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.072706    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b2fcb22-4078-4029-bf55-3076896689ef" containerName="maas-api"
Oct 30 21:40:33.072828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.072819    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b2fcb22-4078-4029-bf55-3076896689ef" containerName="maas-api"
Oct 30 21:40:33.076799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.076783    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:33.093412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.093390    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:40:33.223233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.222682    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-be8b73-dockercfg-rkcf6\""
Oct 30 21:40:33.223233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.222793    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-be8b73\""
Oct 30 21:40:33.249233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.238265    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"]
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.251567    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-envoy\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.251735    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b39fbcd0-dd33-4041-8496-c9310c19856e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b39fbcd0-dd33-4041-8496-c9310c19856e-istiod-ca-cert\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.251782    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-credential-socket\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.251946    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-token\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-token\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.252033    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-kube-api-access-wzsc8\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-kube-api-access-wzsc8\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.252085    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-data\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.252155    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-podinfo\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.252192    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-certs\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.252257    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-socket\") pod \"b39fbcd0-dd33-4041-8496-c9310c19856e\" (UID: \"b39fbcd0-dd33-4041-8496-c9310c19856e\") "
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.252598    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcgcm\" (UniqueName: \"kubernetes.io/projected/91bd3b74-19bf-4b72-b892-dc85d1dec133-kube-api-access-vcgcm\") pod \"payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b\" (UID: \"91bd3b74-19bf-4b72-b892-dc85d1dec133\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:33.261226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.252937    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91bd3b74-19bf-4b72-b892-dc85d1dec133-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b\" (UID: \"91bd3b74-19bf-4b72-b892-dc85d1dec133\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:33.334404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.319299    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-socket" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:33.338872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.338159    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-certs" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:33.353142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.344718    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b39fbcd0-dd33-4041-8496-c9310c19856e-istiod-ca-cert" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:33.384299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.363313    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-data" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:33.384299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.366061    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91bd3b74-19bf-4b72-b892-dc85d1dec133-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b\" (UID: \"91bd3b74-19bf-4b72-b892-dc85d1dec133\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:33.384299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.366304    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcgcm\" (UniqueName: \"kubernetes.io/projected/91bd3b74-19bf-4b72-b892-dc85d1dec133-kube-api-access-vcgcm\") pod \"payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b\" (UID: \"91bd3b74-19bf-4b72-b892-dc85d1dec133\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:33.384299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.376363    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-credential-socket" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:33.384658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.384475    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.384658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.384507    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.384658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.384522    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.384658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.384539    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b39fbcd0-dd33-4041-8496-c9310c19856e-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.384658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.384553    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.449230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.445489    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91bd3b74-19bf-4b72-b892-dc85d1dec133-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b\" (UID: \"91bd3b74-19bf-4b72-b892-dc85d1dec133\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:33.467229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.460274    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"]
Oct 30 21:40:33.499237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.490624    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-kube-api-access-wzsc8" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "kube-api-access-wzsc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:33.499237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.490751    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-envoy" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:33.499237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.491577    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcgcm\" (UniqueName: \"kubernetes.io/projected/91bd3b74-19bf-4b72-b892-dc85d1dec133-kube-api-access-vcgcm\") pod \"payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b\" (UID: \"91bd3b74-19bf-4b72-b892-dc85d1dec133\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:33.499237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.491765    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-token" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:33.499237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.491863    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-podinfo" pod "b39fbcd0-dd33-4041-8496-c9310c19856e" (UID: "b39fbcd0-dd33-4041-8496-c9310c19856e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:40:33.532851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.503800    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.532851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.503877    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.532851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.503915    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzsc8\" (UniqueName: \"kubernetes.io/projected/b39fbcd0-dd33-4041-8496-c9310c19856e-kube-api-access-wzsc8\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.532851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.503932    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b39fbcd0-dd33-4041-8496-c9310c19856e-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.557868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.557838    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy"
Oct 30 21:40:33.557868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.557864    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy"
Oct 30 21:40:33.558094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.557886    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy"
Oct 30 21:40:33.558094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.557894    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy"
Oct 30 21:40:33.558526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.558441    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy"
Oct 30 21:40:33.558526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.558475    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerName="istio-proxy"
Oct 30 21:40:33.562890 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.562860    5786 generic.go:356] "Generic (PLEG): container finished" podID="b39fbcd0-dd33-4041-8496-c9310c19856e" containerID="511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc" exitCode=0
Oct 30 21:40:33.563036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.563026    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"
Oct 30 21:40:33.564390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.564369    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" event={"ID":"b39fbcd0-dd33-4041-8496-c9310c19856e","Type":"ContainerDied","Data":"511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc"}
Oct 30 21:40:33.564495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.564409    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp" event={"ID":"b39fbcd0-dd33-4041-8496-c9310c19856e","Type":"ContainerDied","Data":"24c0665d2230732f959a670e14fd6d34b0b41665dabd1fb55a852b111cad658e"}
Oct 30 21:40:33.564495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.564431    5786 scope.go:122] "RemoveContainer" containerID="511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc"
Oct 30 21:40:33.564495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.564489    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:33.568353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.568332    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"]
Oct 30 21:40:33.590389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.590336    5786 scope.go:122] "RemoveContainer" containerID="ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e"
Oct 30 21:40:33.606122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.606104    5786 scope.go:122] "RemoveContainer" containerID="511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc"
Oct 30 21:40:33.606504 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:40:33.606468    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc\": container with ID starting with 511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc not found: ID does not exist" containerID="511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc"
Oct 30 21:40:33.606596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.606514    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc"} err="failed to get container status \"511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc\": rpc error: code = NotFound desc = could not find container \"511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc\": container with ID starting with 511ee4b53cae1600e557568b5a9e631d7b322276610c2f59d497f777e992a6cc not found: ID does not exist"
Oct 30 21:40:33.606596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.606540    5786 scope.go:122] "RemoveContainer" containerID="ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e"
Oct 30 21:40:33.606824 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:40:33.606803    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e\": container with ID starting with ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e not found: ID does not exist" containerID="ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e"
Oct 30 21:40:33.606863 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.606832    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e"} err="failed to get container status \"ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e\": rpc error: code = NotFound desc = could not find container \"ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e\": container with ID starting with ec2c7cbbbc736b5d93195f1d193c8785583bf782772a4c639db36a5aa87bba2e not found: ID does not exist"
Oct 30 21:40:33.702713 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.702685    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:33.707654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.707626    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf7wb\" (UniqueName: \"kubernetes.io/projected/977c46a1-044a-4545-8181-58c5ba1b0a18-kube-api-access-cf7wb\") pod \"payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs\" (UID: \"977c46a1-044a-4545-8181-58c5ba1b0a18\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:33.707921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.707898    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977c46a1-044a-4545-8181-58c5ba1b0a18-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs\" (UID: \"977c46a1-044a-4545-8181-58c5ba1b0a18\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:33.850354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.817407    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf7wb\" (UniqueName: \"kubernetes.io/projected/977c46a1-044a-4545-8181-58c5ba1b0a18-kube-api-access-cf7wb\") pod \"payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs\" (UID: \"977c46a1-044a-4545-8181-58c5ba1b0a18\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:33.850354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.817606    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977c46a1-044a-4545-8181-58c5ba1b0a18-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs\" (UID: \"977c46a1-044a-4545-8181-58c5ba1b0a18\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:33.850354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.831672    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977c46a1-044a-4545-8181-58c5ba1b0a18-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs\" (UID: \"977c46a1-044a-4545-8181-58c5ba1b0a18\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:33.851850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.851806    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"]
Oct 30 21:40:33.915227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:33.898447    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.025161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.011755    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"]
Oct 30 21:40:34.026499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.025971    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"]
Oct 30 21:40:34.181640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.136403    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-7cf226-openshift-default-dockercfg-gpb2q\""
Oct 30 21:40:34.287026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.272671    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-a7a6ff99-gw-openshift-default-6b66b66657-7zhnp"]
Oct 30 21:40:34.287026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.279029    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf7wb\" (UniqueName: \"kubernetes.io/projected/977c46a1-044a-4545-8181-58c5ba1b0a18-kube-api-access-cf7wb\") pod \"payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs\" (UID: \"977c46a1-044a-4545-8181-58c5ba1b0a18\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:34.336876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.336838    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-certs\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.337239 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.337201    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-credential-socket\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.337465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.337446    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lgdr\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-kube-api-access-6lgdr\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.337671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.337658    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-envoy\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.337783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.337770    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-data\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.337891 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.337881    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-socket\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.337996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.337980    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30a4fbd5-4b68-4316-8e2b-c693757f3009-istiod-ca-cert\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.338120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.338108    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-token\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.338460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.338442    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-podinfo\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.353822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.353790    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"]
Oct 30 21:40:34.354074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.354033    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a0c8aad3-pr5zr"]
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.447867    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-data\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.447918    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-socket\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.447972    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30a4fbd5-4b68-4316-8e2b-c693757f3009-istiod-ca-cert\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.448002    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-token\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.448224    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-podinfo\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.448317    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-certs\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.448438    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-credential-socket\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.448518    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lgdr\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-kube-api-access-6lgdr\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.448669    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-envoy\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.453228    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-credential-socket\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.453307    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-certs\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.453608    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-data\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.453799    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30a4fbd5-4b68-4316-8e2b-c693757f3009-istiod-ca-cert\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.454004    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-socket\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.455532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.455160    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-envoy\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.457522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.457500    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-podinfo\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.459817 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.459790    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b39fbcd0-dd33-4041-8496-c9310c19856e" path="/var/lib/kubelet/pods/b39fbcd0-dd33-4041-8496-c9310c19856e/volumes"
Oct 30 21:40:34.460403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.460382    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c849a498-3ab9-41c2-bc85-5c8ca6ef22e9" path="/var/lib/kubelet/pods/c849a498-3ab9-41c2-bc85-5c8ca6ef22e9/volumes"
Oct 30 21:40:34.497119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.497100    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"]
Oct 30 21:40:34.497917 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:34.497898    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91bd3b74_19bf_4b72_b892_dc85d1dec133.slice/crio-ebca06a6da727fd2bfd789cc628ed5f027a33ac5c1fa6e4b59886fadf57c3488 WatchSource:0}: Error finding container ebca06a6da727fd2bfd789cc628ed5f027a33ac5c1fa6e4b59886fadf57c3488: Status 404 returned error can't find the container with id ebca06a6da727fd2bfd789cc628ed5f027a33ac5c1fa6e4b59886fadf57c3488
Oct 30 21:40:34.506682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.506659    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:34.509050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.509027    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lgdr\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-kube-api-access-6lgdr\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.509577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.509558    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-token\") pod \"e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") " pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.576717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.576689    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b" event={"ID":"91bd3b74-19bf-4b72-b892-dc85d1dec133","Type":"ContainerStarted","Data":"ebca06a6da727fd2bfd789cc628ed5f027a33ac5c1fa6e4b59886fadf57c3488"}
Oct 30 21:40:34.708623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.708558    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:34.898928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:34.898890    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"]
Oct 30 21:40:34.899370 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:34.899347    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977c46a1_044a_4545_8181_58c5ba1b0a18.slice/crio-a9990fc3d8fc5137e3a16d7b5a8b1e27b6bd02dddb247e23e83706a09431f633 WatchSource:0}: Error finding container a9990fc3d8fc5137e3a16d7b5a8b1e27b6bd02dddb247e23e83706a09431f633: Status 404 returned error can't find the container with id a9990fc3d8fc5137e3a16d7b5a8b1e27b6bd02dddb247e23e83706a09431f633
Oct 30 21:40:35.090640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.090614    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"]
Oct 30 21:40:35.091511 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:35.091487    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a4fbd5_4b68_4316_8e2b_c693757f3009.slice/crio-fc8fe615a16f2dfc4613dfc11c7b6d5962ed65830b8445b28ee1020875d2eaf7 WatchSource:0}: Error finding container fc8fe615a16f2dfc4613dfc11c7b6d5962ed65830b8445b28ee1020875d2eaf7: Status 404 returned error can't find the container with id fc8fe615a16f2dfc4613dfc11c7b6d5962ed65830b8445b28ee1020875d2eaf7
Oct 30 21:40:35.094536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.094507    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:40:35.094617 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.094566    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:40:35.094617 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.094591    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:40:35.586883 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.586851    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" event={"ID":"30a4fbd5-4b68-4316-8e2b-c693757f3009","Type":"ContainerStarted","Data":"862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033"}
Oct 30 21:40:35.587366 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.586904    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" event={"ID":"30a4fbd5-4b68-4316-8e2b-c693757f3009","Type":"ContainerStarted","Data":"fc8fe615a16f2dfc4613dfc11c7b6d5962ed65830b8445b28ee1020875d2eaf7"}
Oct 30 21:40:35.588625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.588602    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs" event={"ID":"977c46a1-044a-4545-8181-58c5ba1b0a18","Type":"ContainerStarted","Data":"bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3"}
Oct 30 21:40:35.588715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.588639    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs" event={"ID":"977c46a1-044a-4545-8181-58c5ba1b0a18","Type":"ContainerStarted","Data":"a9990fc3d8fc5137e3a16d7b5a8b1e27b6bd02dddb247e23e83706a09431f633"}
Oct 30 21:40:35.588715 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.588688    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:35.590268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.590248    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b" event={"ID":"91bd3b74-19bf-4b72-b892-dc85d1dec133","Type":"ContainerStarted","Data":"5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e"}
Oct 30 21:40:35.590386 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.590373    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:35.705281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.705244    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" podStartSLOduration=2.7051934109999998 podStartE2EDuration="2.705193411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:35.702230654 +0000 UTC m=+1691.573349772" watchObservedRunningTime="2026-10-30 21:40:35.705193411 +0000 UTC m=+1691.576312527"
Oct 30 21:40:35.708767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:35.708751    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:36.003256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:36.003162    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b" podStartSLOduration=2.998546037 podStartE2EDuration="2.998546037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:35.995382156 +0000 UTC m=+1691.866501277" watchObservedRunningTime="2026-10-30 21:40:35.998546037 +0000 UTC m=+1691.869665157"
Oct 30 21:40:36.011199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:36.007979    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs" podStartSLOduration=3.007961437 podStartE2EDuration="3.007961437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:35.84612256 +0000 UTC m=+1691.717241680" watchObservedRunningTime="2026-10-30 21:40:36.007961437 +0000 UTC m=+1691.879080556"
Oct 30 21:40:36.713512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:36.713485    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:36.959256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:36.959222    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"]
Oct 30 21:40:36.964071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:36.964023    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.016116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.016045    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bf1b02c7-serving-cert\""
Oct 30 21:40:37.021191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.021168    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"]
Oct 30 21:40:37.078517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.078489    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.078727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.078708    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2670cbed-0d37-4982-99e3-e50b3308c23a-openshift-service-ca\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.078776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.078741    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-metrics-certs\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.078902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.078888    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx2tr\" (UniqueName: \"kubernetes.io/projected/2670cbed-0d37-4982-99e3-e50b3308c23a-kube-api-access-vx2tr\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.179859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.179831    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2670cbed-0d37-4982-99e3-e50b3308c23a-openshift-service-ca\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.180030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.179871    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-metrics-certs\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.180095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.180036    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx2tr\" (UniqueName: \"kubernetes.io/projected/2670cbed-0d37-4982-99e3-e50b3308c23a-kube-api-access-vx2tr\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.180095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.180073    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.180298 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:40:37.180277    5786 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-serving-cert: secret "maas-api-e2e-mt-bf1b02c7-serving-cert" not found
Oct 30 21:40:37.180368 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:40:37.180335    5786 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls podName:2670cbed-0d37-4982-99e3-e50b3308c23a nodeName:}" failed. No retries permitted until 2026-10-30 21:40:37.6803195 +0000 UTC m=+1693.551438599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls") pod "maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k" (UID: "2670cbed-0d37-4982-99e3-e50b3308c23a") : secret "maas-api-e2e-mt-bf1b02c7-serving-cert" not found
Oct 30 21:40:37.180569 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.180553    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2670cbed-0d37-4982-99e3-e50b3308c23a-openshift-service-ca\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.182408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.182384    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-metrics-certs\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.265309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.263070    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx2tr\" (UniqueName: \"kubernetes.io/projected/2670cbed-0d37-4982-99e3-e50b3308c23a-kube-api-access-vx2tr\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.609490 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.609405    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:37.610517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.610497    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:40:37.691256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.691225    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.693630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.693601    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls\") pod \"maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:37.879911 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:37.879839    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:38.266690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:38.266663    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"]
Oct 30 21:40:38.267797 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:38.267768    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2670cbed_0d37_4982_99e3_e50b3308c23a.slice/crio-2087cad8e1fe28e5a540ce59afe59f3a4d70fc775cb52d5f84962f8c070b1b00 WatchSource:0}: Error finding container 2087cad8e1fe28e5a540ce59afe59f3a4d70fc775cb52d5f84962f8c070b1b00: Status 404 returned error can't find the container with id 2087cad8e1fe28e5a540ce59afe59f3a4d70fc775cb52d5f84962f8c070b1b00
Oct 30 21:40:38.619609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:38.619509    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k" event={"ID":"2670cbed-0d37-4982-99e3-e50b3308c23a","Type":"ContainerStarted","Data":"2087cad8e1fe28e5a540ce59afe59f3a4d70fc775cb52d5f84962f8c070b1b00"}
Oct 30 21:40:39.631654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:39.631618    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k" event={"ID":"2670cbed-0d37-4982-99e3-e50b3308c23a","Type":"ContainerStarted","Data":"70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad"}
Oct 30 21:40:39.632085 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:39.631690    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:40.126832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:40.126781    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k" podStartSLOduration=2.9572768270000003 podStartE2EDuration="4.126766902s" totalImagesPullingTime="1.169490075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:39.835624643 +0000 UTC m=+1695.706743775" watchObservedRunningTime="2026-10-30 21:40:40.126766902 +0000 UTC m=+1695.997886059"
Oct 30 21:40:40.129336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:40.129308    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"]
Oct 30 21:40:40.137807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:40.137785    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc3dc3ab-fwcj7"]
Oct 30 21:40:40.365980 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:40.365950    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="484f95a6-d51d-43f6-903d-8879bbb0cd2e" path="/var/lib/kubelet/pods/484f95a6-d51d-43f6-903d-8879bbb0cd2e/volumes"
Oct 30 21:40:42.062576 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:42.062542    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"]
Oct 30 21:40:42.110995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:42.110964    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8b170718-zj9tf"]
Oct 30 21:40:42.361653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:42.361566    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0785e604-6eaf-4e7c-be09-90130c919abb" path="/var/lib/kubelet/pods/0785e604-6eaf-4e7c-be09-90130c919abb/volumes"
Oct 30 21:40:43.684142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:43.681388    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"]
Oct 30 21:40:43.772537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:43.765172    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:43.818674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:43.818533    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"]
Oct 30 21:40:43.956051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:43.955979    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-7cf226\""
Oct 30 21:40:44.041307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:43.989494    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-7cf226-dockercfg-5gd2h\""
Oct 30 21:40:44.041524 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.036179    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"]
Oct 30 21:40:44.076583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.076551    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"]
Oct 30 21:40:44.077444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.077420    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:44.145879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.145846    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b5e4b-ca15-40fd-998d-bde2bcbe945c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s\" (UID: \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:44.146642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.146480    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsp4v\" (UniqueName: \"kubernetes.io/projected/889b5e4b-ca15-40fd-998d-bde2bcbe945c-kube-api-access-lsp4v\") pod \"payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s\" (UID: \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:44.248176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.248100    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsp4v\" (UniqueName: \"kubernetes.io/projected/889b5e4b-ca15-40fd-998d-bde2bcbe945c-kube-api-access-lsp4v\") pod \"payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s\" (UID: \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:44.248363 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.248251    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b5e4b-ca15-40fd-998d-bde2bcbe945c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s\" (UID: \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:44.248487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.248465    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c74097eb-e5f3-445c-9df6-c396f9aa9b38-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d\" (UID: \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:44.248773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.248750    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwbdd\" (UniqueName: \"kubernetes.io/projected/c74097eb-e5f3-445c-9df6-c396f9aa9b38-kube-api-access-vwbdd\") pod \"payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d\" (UID: \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:44.249030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.249011    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b5e4b-ca15-40fd-998d-bde2bcbe945c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s\" (UID: \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:44.304196 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.304171    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsp4v\" (UniqueName: \"kubernetes.io/projected/889b5e4b-ca15-40fd-998d-bde2bcbe945c-kube-api-access-lsp4v\") pod \"payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s\" (UID: \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:44.350484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.350456    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwbdd\" (UniqueName: \"kubernetes.io/projected/c74097eb-e5f3-445c-9df6-c396f9aa9b38-kube-api-access-vwbdd\") pod \"payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d\" (UID: \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:44.350842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.350823    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c74097eb-e5f3-445c-9df6-c396f9aa9b38-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d\" (UID: \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:44.351788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.351762    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c74097eb-e5f3-445c-9df6-c396f9aa9b38-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d\" (UID: \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:44.407762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.407728    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwbdd\" (UniqueName: \"kubernetes.io/projected/c74097eb-e5f3-445c-9df6-c396f9aa9b38-kube-api-access-vwbdd\") pod \"payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d\" (UID: \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:44.480148 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.480113    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:44.489214 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.489186    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:44.871374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.871309    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"]
Oct 30 21:40:44.911098 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:44.911073    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889b5e4b_ca15_40fd_998d_bde2bcbe945c.slice/crio-3809551bb062cb6175e2765e4c3f7769a476d4b5454fb32f5f27e99586f8f4f3 WatchSource:0}: Error finding container 3809551bb062cb6175e2765e4c3f7769a476d4b5454fb32f5f27e99586f8f4f3: Status 404 returned error can't find the container with id 3809551bb062cb6175e2765e4c3f7769a476d4b5454fb32f5f27e99586f8f4f3
Oct 30 21:40:44.923821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:44.923796    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"]
Oct 30 21:40:44.929379 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:44.929351    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc74097eb_e5f3_445c_9df6_c396f9aa9b38.slice/crio-b7343269b6a1b8c2ee373f4f19770f14bb50b440ca4d2da4ceb5a992066e967d WatchSource:0}: Error finding container b7343269b6a1b8c2ee373f4f19770f14bb50b440ca4d2da4ceb5a992066e967d: Status 404 returned error can't find the container with id b7343269b6a1b8c2ee373f4f19770f14bb50b440ca4d2da4ceb5a992066e967d
Oct 30 21:40:45.020712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.020686    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s" event={"ID":"889b5e4b-ca15-40fd-998d-bde2bcbe945c","Type":"ContainerStarted","Data":"8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a"}
Oct 30 21:40:45.020844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.020739    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s" event={"ID":"889b5e4b-ca15-40fd-998d-bde2bcbe945c","Type":"ContainerStarted","Data":"3809551bb062cb6175e2765e4c3f7769a476d4b5454fb32f5f27e99586f8f4f3"}
Oct 30 21:40:45.020844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.020773    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:45.022487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.022464    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d" event={"ID":"c74097eb-e5f3-445c-9df6-c396f9aa9b38","Type":"ContainerStarted","Data":"33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1"}
Oct 30 21:40:45.022598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.022518    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d" event={"ID":"c74097eb-e5f3-445c-9df6-c396f9aa9b38","Type":"ContainerStarted","Data":"b7343269b6a1b8c2ee373f4f19770f14bb50b440ca4d2da4ceb5a992066e967d"}
Oct 30 21:40:45.022598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.022545    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:45.144749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.144648    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s" podStartSLOduration=2.144632777 podStartE2EDuration="2.144632777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:45.140120612 +0000 UTC m=+1701.011239735" watchObservedRunningTime="2026-10-30 21:40:45.144632777 +0000 UTC m=+1701.015751892"
Oct 30 21:40:45.285273 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.282025    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d" podStartSLOduration=2.282003914 podStartE2EDuration="2.282003914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:45.279497407 +0000 UTC m=+1701.150616528" watchObservedRunningTime="2026-10-30 21:40:45.282003914 +0000 UTC m=+1701.153123034"
Oct 30 21:40:45.699231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:45.699034    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:40:46.603749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:46.603707    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:40:46.604284 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:46.603783    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:40:50.833244 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:50.823987    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"]
Oct 30 21:40:50.991228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:50.985438    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"]
Oct 30 21:40:50.991228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:50.985575    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:51.031057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.016848    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e08d338-b1fa-49e1-95e6-b816053cddf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s\" (UID: \"7e08d338-b1fa-49e1-95e6-b816053cddf0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:51.031057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.017049    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2hlt\" (UniqueName: \"kubernetes.io/projected/7e08d338-b1fa-49e1-95e6-b816053cddf0-kube-api-access-h2hlt\") pod \"payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s\" (UID: \"7e08d338-b1fa-49e1-95e6-b816053cddf0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:51.127305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.122580    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"]
Oct 30 21:40:51.138272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.128846    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:51.138272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.130459    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-815dab\""
Oct 30 21:40:51.138272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.130528    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-815dab-dockercfg-pqqbv\""
Oct 30 21:40:51.174853 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.172485    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"]
Oct 30 21:40:51.237079 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.212346    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e08d338-b1fa-49e1-95e6-b816053cddf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s\" (UID: \"7e08d338-b1fa-49e1-95e6-b816053cddf0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:51.278231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.265467    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b39f91-0891-47c8-bbff-6659f3bc6998-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb\" (UID: \"64b39f91-0891-47c8-bbff-6659f3bc6998\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:51.278701 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.278661    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2hlt\" (UniqueName: \"kubernetes.io/projected/7e08d338-b1fa-49e1-95e6-b816053cddf0-kube-api-access-h2hlt\") pod \"payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s\" (UID: \"7e08d338-b1fa-49e1-95e6-b816053cddf0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:51.279582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.279549    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz8nt\" (UniqueName: \"kubernetes.io/projected/64b39f91-0891-47c8-bbff-6659f3bc6998-kube-api-access-vz8nt\") pod \"payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb\" (UID: \"64b39f91-0891-47c8-bbff-6659f3bc6998\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:51.280601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.239715    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e08d338-b1fa-49e1-95e6-b816053cddf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s\" (UID: \"7e08d338-b1fa-49e1-95e6-b816053cddf0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:51.378025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.377948    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2hlt\" (UniqueName: \"kubernetes.io/projected/7e08d338-b1fa-49e1-95e6-b816053cddf0-kube-api-access-h2hlt\") pod \"payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s\" (UID: \"7e08d338-b1fa-49e1-95e6-b816053cddf0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:51.383542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.382760    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz8nt\" (UniqueName: \"kubernetes.io/projected/64b39f91-0891-47c8-bbff-6659f3bc6998-kube-api-access-vz8nt\") pod \"payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb\" (UID: \"64b39f91-0891-47c8-bbff-6659f3bc6998\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:51.383542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.382997    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b39f91-0891-47c8-bbff-6659f3bc6998-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb\" (UID: \"64b39f91-0891-47c8-bbff-6659f3bc6998\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:51.383982 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.383954    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b39f91-0891-47c8-bbff-6659f3bc6998-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb\" (UID: \"64b39f91-0891-47c8-bbff-6659f3bc6998\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:51.439429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.439397    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz8nt\" (UniqueName: \"kubernetes.io/projected/64b39f91-0891-47c8-bbff-6659f3bc6998-kube-api-access-vz8nt\") pod \"payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb\" (UID: \"64b39f91-0891-47c8-bbff-6659f3bc6998\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:51.662555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.662484    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:51.671056 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:51.671027    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:52.051246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:52.051177    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"]
Oct 30 21:40:52.051654 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:52.051441    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b39f91_0891_47c8_bbff_6659f3bc6998.slice/crio-16953dac0532964431806b17ecdc2203e1558a984781a747e034bcc1ea747611 WatchSource:0}: Error finding container 16953dac0532964431806b17ecdc2203e1558a984781a747e034bcc1ea747611: Status 404 returned error can't find the container with id 16953dac0532964431806b17ecdc2203e1558a984781a747e034bcc1ea747611
Oct 30 21:40:52.101942 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:52.101915    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb" event={"ID":"64b39f91-0891-47c8-bbff-6659f3bc6998","Type":"ContainerStarted","Data":"16953dac0532964431806b17ecdc2203e1558a984781a747e034bcc1ea747611"}
Oct 30 21:40:52.102881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:52.102860    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"]
Oct 30 21:40:52.104269 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:40:52.104246    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e08d338_b1fa_49e1_95e6_b816053cddf0.slice/crio-b90b563a14931ed4c17e1a2894b14c5b6ba4c30933e4bf5c87c847e1621a2f39 WatchSource:0}: Error finding container b90b563a14931ed4c17e1a2894b14c5b6ba4c30933e4bf5c87c847e1621a2f39: Status 404 returned error can't find the container with id b90b563a14931ed4c17e1a2894b14c5b6ba4c30933e4bf5c87c847e1621a2f39
Oct 30 21:40:52.360954 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:40:52.360881    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:40:53.201545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:53.201499    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb" event={"ID":"64b39f91-0891-47c8-bbff-6659f3bc6998","Type":"ContainerStarted","Data":"cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f"}
Oct 30 21:40:53.202043 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:53.201961    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:40:53.202969 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:53.202930    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s" event={"ID":"7e08d338-b1fa-49e1-95e6-b816053cddf0","Type":"ContainerStarted","Data":"5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab"}
Oct 30 21:40:53.203178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:53.203159    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s" event={"ID":"7e08d338-b1fa-49e1-95e6-b816053cddf0","Type":"ContainerStarted","Data":"b90b563a14931ed4c17e1a2894b14c5b6ba4c30933e4bf5c87c847e1621a2f39"}
Oct 30 21:40:53.203350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:53.203335    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:40:53.318787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:53.318737    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb" podStartSLOduration=3.318726095 podStartE2EDuration="3.318726095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:53.31705568 +0000 UTC m=+1709.188174824" watchObservedRunningTime="2026-10-30 21:40:53.318726095 +0000 UTC m=+1709.189845213"
Oct 30 21:40:56.033390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:56.033360    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:40:56.033801 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:56.033578    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:40:56.151241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:40:56.151183    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s" podStartSLOduration=6.151170688 podStartE2EDuration="6.151170688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:53.463111923 +0000 UTC m=+1709.334231045" watchObservedRunningTime="2026-10-30 21:40:56.151170688 +0000 UTC m=+1712.022289855"
Oct 30 21:41:00.052266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:00.052178    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"]
Oct 30 21:41:00.101462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:00.101435    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a88fe2c3-v7cg4"]
Oct 30 21:41:00.361182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:00.361112    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea826efb-93df-4ca2-8a61-f2036c25d462" path="/var/lib/kubelet/pods/ea826efb-93df-4ca2-8a61-f2036c25d462/volumes"
Oct 30 21:41:02.051983 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:02.051952    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"]
Oct 30 21:41:02.100628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:02.100602    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aac4ddbb-htdpm"]
Oct 30 21:41:02.361003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:02.360931    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95ff2755-4034-4522-b09d-6b46fe8f5a63" path="/var/lib/kubelet/pods/95ff2755-4034-4522-b09d-6b46fe8f5a63/volumes"
Oct 30 21:41:04.213465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:04.213436    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:41:04.213945 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:04.213510    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:41:12.330634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.330599    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"]
Oct 30 21:41:12.331082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.330809    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy" containerID="cri-o://b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8" gracePeriod=30
Oct 30 21:41:12.467184 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.467145    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"]
Oct 30 21:41:12.467398 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.467369    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy" containerID="cri-o://b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215" gracePeriod=30
Oct 30 21:41:12.503262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.503230    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"]
Oct 30 21:41:12.503476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.503446    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy" containerID="cri-o://64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97" gracePeriod=30
Oct 30 21:41:12.509395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.508979    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"]
Oct 30 21:41:12.509395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.509193    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy" containerID="cri-o://c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88" gracePeriod=30
Oct 30 21:41:12.513756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.513732    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"]
Oct 30 21:41:12.514057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:12.514028    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy" containerID="cri-o://5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079" gracePeriod=30
Oct 30 21:41:17.726834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.726477    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:41:17.731298 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:17.731259    5786 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a92f2bb_b279_427b_b8bc_9143dfcd6cff.slice/crio-conmon-64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3760152e_c736_402c_89b3_b862b8fd1193.slice/crio-c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:41:17.760754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.760730    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-kube-api-access-lhxdg\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-kube-api-access-lhxdg\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.760917 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.760790    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-data\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.760917 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.760853    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/040f7639-1a65-4901-bda0-8f7c852045a3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040f7639-1a65-4901-bda0-8f7c852045a3-istiod-ca-cert\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.760917 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.760885    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-certs\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.761072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761013    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-envoy\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.761135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761077    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-credential-socket\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.761198 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761133    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-data" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.761198 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761157    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-istio-token\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-istio-token\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.761198 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761190    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-socket\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.761409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761365    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/040f7639-1a65-4901-bda0-8f7c852045a3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040f7639-1a65-4901-bda0-8f7c852045a3-istio-podinfo\") pod \"040f7639-1a65-4901-bda0-8f7c852045a3\" (UID: \"040f7639-1a65-4901-bda0-8f7c852045a3\") "
Oct 30 21:41:17.761409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761370    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-credential-socket" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.761517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761483    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040f7639-1a65-4901-bda0-8f7c852045a3-istiod-ca-cert" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:17.761607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761589    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-socket" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.761783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.761746    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-certs" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.762297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.762275    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.762379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.762305    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040f7639-1a65-4901-bda0-8f7c852045a3-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.762379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.762319    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.762379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.762332    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.762379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.762348    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.764494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.764470    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/040f7639-1a65-4901-bda0-8f7c852045a3-istio-podinfo" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:17.764623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.764590    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-kube-api-access-lhxdg" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "kube-api-access-lhxdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.765055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.765031    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-envoy" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.765632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.765601    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-istio-token" pod "040f7639-1a65-4901-bda0-8f7c852045a3" (UID: "040f7639-1a65-4901-bda0-8f7c852045a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.776702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.776682    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:41:17.848610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.848547    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:41:17.863880 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.863849    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-certs\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.864031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.863913    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/688684ad-2f63-44f1-9855-636878fecb22-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/688684ad-2f63-44f1-9855-636878fecb22-istio-podinfo\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.864285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.864260    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-envoy\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.864375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.864340    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-istio-token\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-istio-token\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.864457 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.864441    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-credential-socket\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.864521 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.864504    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-data\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.864789 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.864740    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-kube-api-access-4lx66\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-kube-api-access-4lx66\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.864972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.864957    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-socket\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.865034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.865013    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/688684ad-2f63-44f1-9855-636878fecb22-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/688684ad-2f63-44f1-9855-636878fecb22-istiod-ca-cert\") pod \"688684ad-2f63-44f1-9855-636878fecb22\" (UID: \"688684ad-2f63-44f1-9855-636878fecb22\") "
Oct 30 21:41:17.865105 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.865076    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-data" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.865591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.865564    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-socket" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.865686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.865653    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/688684ad-2f63-44f1-9855-636878fecb22-istiod-ca-cert" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:17.865686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.865663    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-credential-socket" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.865894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.865874    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-certs" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.866033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.866015    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.866149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.866041    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/688684ad-2f63-44f1-9855-636878fecb22-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.866149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.866056    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040f7639-1a65-4901-bda0-8f7c852045a3-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.866149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.866070    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.866149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.866085    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040f7639-1a65-4901-bda0-8f7c852045a3-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.866149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.866099    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.866149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.866113    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.866149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.866128    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhxdg\" (UniqueName: \"kubernetes.io/projected/040f7639-1a65-4901-bda0-8f7c852045a3-kube-api-access-lhxdg\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.869484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.869449    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-istio-token" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.870549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.870505    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/688684ad-2f63-44f1-9855-636878fecb22-istio-podinfo" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:17.871387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.871366    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-kube-api-access-4lx66" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "kube-api-access-4lx66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.872493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.872461    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-envoy" pod "688684ad-2f63-44f1-9855-636878fecb22" (UID: "688684ad-2f63-44f1-9855-636878fecb22"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.923766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.923741    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:41:17.925884 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.925865    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:41:17.967399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967372    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-socket\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.967537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967429    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3760152e-c736-402c-89b3-b862b8fd1193-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3760152e-c736-402c-89b3-b862b8fd1193-istiod-ca-cert\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.967537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967459    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-envoy\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.967537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967486    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-socket\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.967705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967565    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-kube-api-access-zr72s\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-kube-api-access-zr72s\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.967705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967654    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-socket" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.967705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967686    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-token\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-token\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.967866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967712    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-socket\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.967866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967759    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-data\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.967866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967787    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istiod-ca-cert\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.967866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967819    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-istio-token\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-istio-token\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.967866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967846    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-data\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.968168 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967898    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-socket" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.968168 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968125    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-data" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.968334 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.967885    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-token\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-token\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.968395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968336    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-certs\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.968395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968368    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-podinfo\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.968501 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968401    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3760152e-c736-402c-89b3-b862b8fd1193-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3760152e-c736-402c-89b3-b862b8fd1193-istio-podinfo\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.968501 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968428    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-credential-socket\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.968501 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968485    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-kube-api-access-zt6tn\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-kube-api-access-zt6tn\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.968663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968518    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-socket" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.968663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968541    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-podinfo\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.968663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968611    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-envoy\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.968832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968676    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-envoy\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.968832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968760    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3760152e-c736-402c-89b3-b862b8fd1193-istiod-ca-cert" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:17.968941 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968861    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-credential-socket\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.968941 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968888    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istiod-ca-cert\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.968941 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968927    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-kube-api-access-p8bk2\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-kube-api-access-p8bk2\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.969092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.968961    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-data\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.969150 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.969087    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-certs\") pod \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\" (UID: \"1a92f2bb-b279-427b-b8bc-9143dfcd6cff\") "
Oct 30 21:41:17.969232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.969168    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istiod-ca-cert" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:17.969232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.969182    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-credential-socket\") pod \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\" (UID: \"b15a6df7-ab5b-4b57-8a78-0e54a806e305\") "
Oct 30 21:41:17.969502 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.969253    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-certs\") pod \"3760152e-c736-402c-89b3-b862b8fd1193\" (UID: \"3760152e-c736-402c-89b3-b862b8fd1193\") "
Oct 30 21:41:17.970266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970242    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970272    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3760152e-c736-402c-89b3-b862b8fd1193-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970287    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970303    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lx66\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-kube-api-access-4lx66\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970319    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970334    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970347    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970361    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970376    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/688684ad-2f63-44f1-9855-636878fecb22-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970389    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/688684ad-2f63-44f1-9855-636878fecb22-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970403    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/688684ad-2f63-44f1-9855-636878fecb22-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:17.970787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970599    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-certs" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.970953 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.970847    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-credential-socket" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.971127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.971105    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istiod-ca-cert" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:17.971221 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.971143    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-token" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.972187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.971544    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-certs" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.972187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.971917    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-data" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.972600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.972577    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-podinfo" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:17.972991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.972776    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-certs" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.972991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.972849    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-envoy" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.972991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.972854    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-istio-token" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.973506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.973148    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-envoy" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.973506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.973361    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-kube-api-access-p8bk2" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "kube-api-access-p8bk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.973506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.973397    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-data" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.973706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.973599    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-credential-socket" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.973764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.973699    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-kube-api-access-zt6tn" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "kube-api-access-zt6tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.973830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.973805    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-credential-socket" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:17.974344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.974316    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-kube-api-access-zr72s" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "kube-api-access-zr72s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.974544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.974521    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-token" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:17.974805 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.974785    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-podinfo" pod "1a92f2bb-b279-427b-b8bc-9143dfcd6cff" (UID: "1a92f2bb-b279-427b-b8bc-9143dfcd6cff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:17.974956 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.974869    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3760152e-c736-402c-89b3-b862b8fd1193-istio-podinfo" pod "3760152e-c736-402c-89b3-b862b8fd1193" (UID: "3760152e-c736-402c-89b3-b862b8fd1193"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:17.974956 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:17.974933    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-envoy" pod "b15a6df7-ab5b-4b57-8a78-0e54a806e305" (UID: "b15a6df7-ab5b-4b57-8a78-0e54a806e305"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:18.072484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072448    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072480    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072494    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072507    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr72s\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-kube-api-access-zr72s\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072521    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072532    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072543    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072555    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072566    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072578    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072592    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3760152e-c736-402c-89b3-b862b8fd1193-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072604    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072618    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt6tn\" (UniqueName: \"kubernetes.io/projected/3760152e-c736-402c-89b3-b862b8fd1193-kube-api-access-zt6tn\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072630    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072641    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15a6df7-ab5b-4b57-8a78-0e54a806e305-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072653    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072664    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3760152e-c736-402c-89b3-b862b8fd1193-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072677    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072693    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8bk2\" (UniqueName: \"kubernetes.io/projected/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-kube-api-access-p8bk2\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072706    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.072732 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.072720    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a92f2bb-b279-427b-b8bc-9143dfcd6cff-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:18.446679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.446643    5786 generic.go:356] "Generic (PLEG): container finished" podID="040f7639-1a65-4901-bda0-8f7c852045a3" containerID="b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8" exitCode=0
Oct 30 21:41:18.446861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.446711    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"
Oct 30 21:41:18.446861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.446752    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" event={"ID":"040f7639-1a65-4901-bda0-8f7c852045a3","Type":"ContainerDied","Data":"b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8"}
Oct 30 21:41:18.446861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.446799    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z" event={"ID":"040f7639-1a65-4901-bda0-8f7c852045a3","Type":"ContainerDied","Data":"67b71698e57c2184cd625692f69ea398498416a90f410ea0ed1274a4fc84a1ad"}
Oct 30 21:41:18.447744 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.447719    5786 scope.go:122] "RemoveContainer" containerID="b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8"
Oct 30 21:41:18.449725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.449704    5786 generic.go:356] "Generic (PLEG): container finished" podID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerID="5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079" exitCode=0
Oct 30 21:41:18.449816 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.449803    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" event={"ID":"b15a6df7-ab5b-4b57-8a78-0e54a806e305","Type":"ContainerDied","Data":"5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079"}
Oct 30 21:41:18.449875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.449844    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c" event={"ID":"b15a6df7-ab5b-4b57-8a78-0e54a806e305","Type":"ContainerDied","Data":"16d7961cc814f1ca6bfac4407e4857cc3aa7b37f5b78edc9e96270a5cce3290a"}
Oct 30 21:41:18.449931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.449916    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"
Oct 30 21:41:18.453690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.453664    5786 generic.go:356] "Generic (PLEG): container finished" podID="3760152e-c736-402c-89b3-b862b8fd1193" containerID="c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88" exitCode=0
Oct 30 21:41:18.453814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.453782    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" event={"ID":"3760152e-c736-402c-89b3-b862b8fd1193","Type":"ContainerDied","Data":"c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88"}
Oct 30 21:41:18.454051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.454026    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2" event={"ID":"3760152e-c736-402c-89b3-b862b8fd1193","Type":"ContainerDied","Data":"064a94e6973e7a68c99f19f5e4fe3196145baca559935838e86a3d21f91bdeba"}
Oct 30 21:41:18.454146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.454131    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"
Oct 30 21:41:18.459265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.457262    5786 generic.go:356] "Generic (PLEG): container finished" podID="688684ad-2f63-44f1-9855-636878fecb22" containerID="b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215" exitCode=0
Oct 30 21:41:18.459265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.457518    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" event={"ID":"688684ad-2f63-44f1-9855-636878fecb22","Type":"ContainerDied","Data":"b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215"}
Oct 30 21:41:18.459265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.457560    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2" event={"ID":"688684ad-2f63-44f1-9855-636878fecb22","Type":"ContainerDied","Data":"5d992c0327920bc6d7440d440993e1af08ebec64e39baaa28fa555cc454aeab6"}
Oct 30 21:41:18.459265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.457629    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"
Oct 30 21:41:18.460684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.460656    5786 generic.go:356] "Generic (PLEG): container finished" podID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerID="64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97" exitCode=0
Oct 30 21:41:18.460794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.460728    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" event={"ID":"1a92f2bb-b279-427b-b8bc-9143dfcd6cff","Type":"ContainerDied","Data":"64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97"}
Oct 30 21:41:18.460794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.460764    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw" event={"ID":"1a92f2bb-b279-427b-b8bc-9143dfcd6cff","Type":"ContainerDied","Data":"2e700c0c69f8f70c3abdd4b6f5904979a50f0e2a7138712559a3cf0378eaadfa"}
Oct 30 21:41:18.460961 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.460945    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"
Oct 30 21:41:18.471311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.471278    5786 scope.go:122] "RemoveContainer" containerID="b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8"
Oct 30 21:41:18.471707 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:18.471671    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8\": container with ID starting with b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8 not found: ID does not exist" containerID="b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8"
Oct 30 21:41:18.471836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.471710    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8"} err="failed to get container status \"b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8\": rpc error: code = NotFound desc = could not find container \"b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8\": container with ID starting with b74ce282348c4f8cd7984e2fc815fcc0db92d09e11f409d52284e6800a7681a8 not found: ID does not exist"
Oct 30 21:41:18.471836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.471735    5786 scope.go:122] "RemoveContainer" containerID="5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079"
Oct 30 21:41:18.490473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.489998    5786 scope.go:122] "RemoveContainer" containerID="5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079"
Oct 30 21:41:18.490473 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:18.490359    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079\": container with ID starting with 5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079 not found: ID does not exist" containerID="5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079"
Oct 30 21:41:18.490473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.490393    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079"} err="failed to get container status \"5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079\": rpc error: code = NotFound desc = could not find container \"5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079\": container with ID starting with 5b036540057a607f6a8aba1e0df6fb233086d3a89b23123d26e34ed6bfbf2079 not found: ID does not exist"
Oct 30 21:41:18.490473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.490416    5786 scope.go:122] "RemoveContainer" containerID="c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88"
Oct 30 21:41:18.503691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.503662    5786 scope.go:122] "RemoveContainer" containerID="eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa"
Oct 30 21:41:18.513880 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.513858    5786 scope.go:122] "RemoveContainer" containerID="c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88"
Oct 30 21:41:18.514185 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:18.514164    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88\": container with ID starting with c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88 not found: ID does not exist" containerID="c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88"
Oct 30 21:41:18.514286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.514189    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88"} err="failed to get container status \"c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88\": rpc error: code = NotFound desc = could not find container \"c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88\": container with ID starting with c5a4d17bc3b559224de117a9d8f3349a5b1845c76d4f81dc11beeaa2661d2c88 not found: ID does not exist"
Oct 30 21:41:18.514286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.514250    5786 scope.go:122] "RemoveContainer" containerID="eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa"
Oct 30 21:41:18.514552 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:18.514525    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa\": container with ID starting with eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa not found: ID does not exist" containerID="eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa"
Oct 30 21:41:18.514650 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.514559    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa"} err="failed to get container status \"eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa\": rpc error: code = NotFound desc = could not find container \"eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa\": container with ID starting with eeb5501ceed9b919957526eeaaafc44319d3d07d3526a40f9d7b44b87c1d57aa not found: ID does not exist"
Oct 30 21:41:18.514650 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.514578    5786 scope.go:122] "RemoveContainer" containerID="b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215"
Oct 30 21:41:18.526677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.526661    5786 scope.go:122] "RemoveContainer" containerID="b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215"
Oct 30 21:41:18.526929 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:18.526912    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215\": container with ID starting with b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215 not found: ID does not exist" containerID="b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215"
Oct 30 21:41:18.526998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.526935    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215"} err="failed to get container status \"b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215\": rpc error: code = NotFound desc = could not find container \"b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215\": container with ID starting with b1ec26eb51a5011a54cb98c73f5b73fd46c172ce4378e28b8bb4255eb812a215 not found: ID does not exist"
Oct 30 21:41:18.526998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.526951    5786 scope.go:122] "RemoveContainer" containerID="64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97"
Oct 30 21:41:18.536602 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.536585    5786 scope.go:122] "RemoveContainer" containerID="64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97"
Oct 30 21:41:18.536849 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:18.536830    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97\": container with ID starting with 64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97 not found: ID does not exist" containerID="64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97"
Oct 30 21:41:18.536904 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.536855    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97"} err="failed to get container status \"64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97\": rpc error: code = NotFound desc = could not find container \"64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97\": container with ID starting with 64e09c46bbc0c8ee28877cdce33d24a276f844828297644021c36164f98feb97 not found: ID does not exist"
Oct 30 21:41:18.617244 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.617141    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"]
Oct 30 21:41:18.622607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.622585    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-fxp6z"]
Oct 30 21:41:18.823231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.822624    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"]
Oct 30 21:41:18.823231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.822688    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-kq7w2"]
Oct 30 21:41:18.890821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.890746    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"]
Oct 30 21:41:18.897019 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:18.896998    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-gngnw"]
Oct 30 21:41:19.012302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:19.012273    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"]
Oct 30 21:41:19.025393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:19.025367    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-v8s7c"]
Oct 30 21:41:19.136356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:19.136322    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"]
Oct 30 21:41:19.142345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:19.142278    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-38b832-openshift-default-849f7b9d8f-6nln2"]
Oct 30 21:41:20.362692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:20.362662    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" path="/var/lib/kubelet/pods/040f7639-1a65-4901-bda0-8f7c852045a3/volumes"
Oct 30 21:41:20.363125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:20.363111    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" path="/var/lib/kubelet/pods/1a92f2bb-b279-427b-b8bc-9143dfcd6cff/volumes"
Oct 30 21:41:20.363563 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:20.363550    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3760152e-c736-402c-89b3-b862b8fd1193" path="/var/lib/kubelet/pods/3760152e-c736-402c-89b3-b862b8fd1193/volumes"
Oct 30 21:41:20.364333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:20.364319    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="688684ad-2f63-44f1-9855-636878fecb22" path="/var/lib/kubelet/pods/688684ad-2f63-44f1-9855-636878fecb22/volumes"
Oct 30 21:41:20.364750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:20.364738    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" path="/var/lib/kubelet/pods/b15a6df7-ab5b-4b57-8a78-0e54a806e305/volumes"
Oct 30 21:41:22.355515 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:22.355486    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:41:26.090083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.090051    5786 scope.go:122] "RemoveContainer" containerID="48a79917a7b51ba7ecadb448f7bd8bc2df5fe297011b2f2af6c11756caea9576"
Oct 30 21:41:26.103860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.103824    5786 scope.go:122] "RemoveContainer" containerID="9b4832b48c578a90e19b8c4ad71c114d1b57d15d66157d1a5188a9336a8cc046"
Oct 30 21:41:26.126717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.126691    5786 scope.go:122] "RemoveContainer" containerID="4f7b71f2c233a832c3843b9240a874bdb546bdab028d3d0e5436020d738780ec"
Oct 30 21:41:26.136879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.136860    5786 scope.go:122] "RemoveContainer" containerID="6dd3f36977af90591511a21dca2b0d039059cacf2b1204e9c082bfebbd952741"
Oct 30 21:41:26.146296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.146264    5786 scope.go:122] "RemoveContainer" containerID="fd59910f22ca58cb6b2efe10ed90f681217162b81e45a1180c3f3058cf60582b"
Oct 30 21:41:26.155236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.155190    5786 scope.go:122] "RemoveContainer" containerID="ca27ec9e5346ab87d79725a757cd4ee49868998ff6be491f50719bdd11cb9f4f"
Oct 30 21:41:26.174829 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.174727    5786 scope.go:122] "RemoveContainer" containerID="baf7360860900944b800cfbb5d4ec4c31471adb1491829cfd7813e42728c8ecf"
Oct 30 21:41:26.185686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.185667    5786 scope.go:122] "RemoveContainer" containerID="92879e6f1ef287fb38c87614f62668df1f742bca10bf601bcc58e6dd18d41af0"
Oct 30 21:41:26.197003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.196984    5786 scope.go:122] "RemoveContainer" containerID="31a810d433bcf64b1b073b954dea4377a71238adb145a205696c8da925fb20d1"
Oct 30 21:41:26.206537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.206518    5786 scope.go:122] "RemoveContainer" containerID="7c32302b4ef0503aed324f60f2bdd60a1736e22d5303f5c1992d6dd8f2db3bb8"
Oct 30 21:41:26.215474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.215454    5786 scope.go:122] "RemoveContainer" containerID="d9d972c8c0033fc4ac5f95447ce7ce468cdfe80b05d7d8291744989ded7d38fb"
Oct 30 21:41:26.224349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:26.224331    5786 scope.go:122] "RemoveContainer" containerID="097ebec7f1ef1d2e34ffb2440b34bdd17b6534670dbb2bd92bd3605b629d062f"
Oct 30 21:41:27.371251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:27.371184    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"]
Oct 30 21:41:27.371691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:27.371442    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy" containerID="cri-o://6e706c11b5e0218148e147526608cc74a3c4ba0780e54df398d5bebfc87adeef" gracePeriod=30
Oct 30 21:41:27.424023 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:27.423996    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"]
Oct 30 21:41:27.424254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:27.424196    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy" containerID="cri-o://daabe7e7bee711ab9e037238787dd3c111e9d0fb4c12970017598ff3f8b45bf7" gracePeriod=30
Oct 30 21:41:27.429460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:27.429434    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"]
Oct 30 21:41:27.432452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:27.429642    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy" containerID="cri-o://6c1de686e80d399060017c1a4ac7d6003eedbff0e6d4845ce6a4a69865c79746" gracePeriod=30
Oct 30 21:41:29.821778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:29.821744    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:41:29.822144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:29.821825    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:41:32.280705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.280675    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"]
Oct 30 21:41:32.639604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.639426    5786 generic.go:356] "Generic (PLEG): container finished" podID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerID="daabe7e7bee711ab9e037238787dd3c111e9d0fb4c12970017598ff3f8b45bf7" exitCode=0
Oct 30 21:41:32.639604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.639504    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" event={"ID":"6ec004d3-0158-47fa-87e1-b21eb07a8938","Type":"ContainerDied","Data":"daabe7e7bee711ab9e037238787dd3c111e9d0fb4c12970017598ff3f8b45bf7"}
Oct 30 21:41:32.641314 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.641289    5786 generic.go:356] "Generic (PLEG): container finished" podID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerID="6c1de686e80d399060017c1a4ac7d6003eedbff0e6d4845ce6a4a69865c79746" exitCode=0
Oct 30 21:41:32.641438 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.641397    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" event={"ID":"f060ad2c-248a-4f3a-a866-1ac2d890bbfd","Type":"ContainerDied","Data":"6c1de686e80d399060017c1a4ac7d6003eedbff0e6d4845ce6a4a69865c79746"}
Oct 30 21:41:32.642987 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.642970    5786 generic.go:356] "Generic (PLEG): container finished" podID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerID="6e706c11b5e0218148e147526608cc74a3c4ba0780e54df398d5bebfc87adeef" exitCode=0
Oct 30 21:41:32.643043 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.643003    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" event={"ID":"7765c1e6-c88f-4863-89ab-a19648eed81e","Type":"ContainerDied","Data":"6e706c11b5e0218148e147526608cc74a3c4ba0780e54df398d5bebfc87adeef"}
Oct 30 21:41:32.721859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.721826    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:41:32.747307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.747271    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:41:32.762149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762122    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-data\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.762300 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762279    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-credential-socket\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.762370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762351    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-token\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-token\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.762406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762379    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-data" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.762452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762438    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-podinfo\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.762487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762469    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-socket\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.762530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762511    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7765c1e6-c88f-4863-89ab-a19648eed81e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7765c1e6-c88f-4863-89ab-a19648eed81e-istiod-ca-cert\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.762571 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762548    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-credential-socket" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.762571 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762560    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-kube-api-access-wbz76\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-kube-api-access-wbz76\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.762632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.762619    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-envoy\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.763694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.763672    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7765c1e6-c88f-4863-89ab-a19648eed81e-istiod-ca-cert" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:32.763895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.763877    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-socket" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.764899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.764865    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-podinfo" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:32.765265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.765243    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-token" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:32.765905 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.765880    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-kube-api-access-wbz76" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "kube-api-access-wbz76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:32.766928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.766909    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-envoy" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.769473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.769450    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-certs\") pod \"7765c1e6-c88f-4863-89ab-a19648eed81e\" (UID: \"7765c1e6-c88f-4863-89ab-a19648eed81e\") "
Oct 30 21:41:32.770238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770198    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.770307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770241    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.770307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770254    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.770307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770267    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.770307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770280    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.770307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770292    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7765c1e6-c88f-4863-89ab-a19648eed81e-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.770307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770308    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbz76\" (UniqueName: \"kubernetes.io/projected/7765c1e6-c88f-4863-89ab-a19648eed81e-kube-api-access-wbz76\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.770519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770322    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.770519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.770412    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-certs" pod "7765c1e6-c88f-4863-89ab-a19648eed81e" (UID: "7765c1e6-c88f-4863-89ab-a19648eed81e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.787731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.787711    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:41:32.871682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.871612    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-envoy\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.871682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.871673    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ec004d3-0158-47fa-87e1-b21eb07a8938-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec004d3-0158-47fa-87e1-b21eb07a8938-istiod-ca-cert\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.871895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.871734    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-envoy\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.871895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.871877    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-data\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.872010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.871918    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-data\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.872010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.871989    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-token\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-token\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.872119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872042    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-podinfo\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.872119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872093    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-socket\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.872119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872103    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-data" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.872312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872134    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ec004d3-0158-47fa-87e1-b21eb07a8938-istiod-ca-cert" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:32.872312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872246    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-kube-api-access-fnbjn\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-kube-api-access-fnbjn\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.872431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872316    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-credential-socket\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.872431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872360    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istiod-ca-cert\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.872431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872401    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-socket\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.872741 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872475    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-socket" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.872741 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872650    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-socket" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.872741 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872726    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-credential-socket" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.872897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872846    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-podinfo\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.872897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872892    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-certs\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.873006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872958    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-credential-socket\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.873060 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.873009    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-kube-api-access-vnx95\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-kube-api-access-vnx95\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.873115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.873056    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-token\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-token\") pod \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\" (UID: \"f060ad2c-248a-4f3a-a866-1ac2d890bbfd\") "
Oct 30 21:41:32.873115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.873093    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-certs\") pod \"6ec004d3-0158-47fa-87e1-b21eb07a8938\" (UID: \"6ec004d3-0158-47fa-87e1-b21eb07a8938\") "
Oct 30 21:41:32.873428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872846    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-data" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.873428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.872972    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istiod-ca-cert" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:32.873558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.873401    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-certs" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.873659 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.873636    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-certs" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.874268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874242    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874272    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874288    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874301    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874314    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874328    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec004d3-0158-47fa-87e1-b21eb07a8938-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874341    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874354    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874368    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874382    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7765c1e6-c88f-4863-89ab-a19648eed81e-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.874686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874583    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-credential-socket" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.874686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.874603    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-envoy" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.876180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.876150    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-podinfo" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:32.876333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.876188    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-token" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:32.876333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.876255    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-envoy" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:32.876538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.876520    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-podinfo" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:32.876623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.876606    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-token" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:32.876623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.876611    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-kube-api-access-vnx95" pod "f060ad2c-248a-4f3a-a866-1ac2d890bbfd" (UID: "f060ad2c-248a-4f3a-a866-1ac2d890bbfd"). InnerVolumeSpecName "kube-api-access-vnx95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:32.876727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.876713    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-kube-api-access-fnbjn" pod "6ec004d3-0158-47fa-87e1-b21eb07a8938" (UID: "6ec004d3-0158-47fa-87e1-b21eb07a8938"). InnerVolumeSpecName "kube-api-access-fnbjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:32.976110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976074    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.976110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976113    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.976418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976129    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnx95\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-kube-api-access-vnx95\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.976418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976145    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.976418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976158    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f060ad2c-248a-4f3a-a866-1ac2d890bbfd-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.976418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976172    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.976418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976186    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.976418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976199    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec004d3-0158-47fa-87e1-b21eb07a8938-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:32.976418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:32.976234    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnbjn\" (UniqueName: \"kubernetes.io/projected/6ec004d3-0158-47fa-87e1-b21eb07a8938-kube-api-access-fnbjn\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:33.652898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.652816    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"
Oct 30 21:41:33.652898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.652851    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt" event={"ID":"f060ad2c-248a-4f3a-a866-1ac2d890bbfd","Type":"ContainerDied","Data":"672402aa82902259b854849bc9f94a976f536765608c19bc9703866ef70433ea"}
Oct 30 21:41:33.653399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.652924    5786 scope.go:122] "RemoveContainer" containerID="6c1de686e80d399060017c1a4ac7d6003eedbff0e6d4845ce6a4a69865c79746"
Oct 30 21:41:33.654719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.654696    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"
Oct 30 21:41:33.654844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.654752    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t" event={"ID":"7765c1e6-c88f-4863-89ab-a19648eed81e","Type":"ContainerDied","Data":"2d7511fb2ca7ea49f70310d9ae16f8b9aa46617fcc54740616ecf06f36500f58"}
Oct 30 21:41:33.656477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.656450    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"
Oct 30 21:41:33.656598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.656505    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d" event={"ID":"6ec004d3-0158-47fa-87e1-b21eb07a8938","Type":"ContainerDied","Data":"4d2480d1635a1ec46a18d94515e13c69c828e8a75d03752e18da9e96631a4ddf"}
Oct 30 21:41:33.666745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.666730    5786 scope.go:122] "RemoveContainer" containerID="6e706c11b5e0218148e147526608cc74a3c4ba0780e54df398d5bebfc87adeef"
Oct 30 21:41:33.676038 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.676020    5786 scope.go:122] "RemoveContainer" containerID="daabe7e7bee711ab9e037238787dd3c111e9d0fb4c12970017598ff3f8b45bf7"
Oct 30 21:41:33.817290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.817260    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"]
Oct 30 21:41:33.823613 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.823589    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-4cc8d"]
Oct 30 21:41:33.933458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.933423    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"]
Oct 30 21:41:33.936608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:33.936589    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-xgv2t"]
Oct 30 21:41:34.043528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:34.043506    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"]
Oct 30 21:41:34.049183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:34.049163    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-t5qvt"]
Oct 30 21:41:34.362889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:34.362822    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" path="/var/lib/kubelet/pods/6ec004d3-0158-47fa-87e1-b21eb07a8938/volumes"
Oct 30 21:41:34.363301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:34.363285    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" path="/var/lib/kubelet/pods/7765c1e6-c88f-4863-89ab-a19648eed81e/volumes"
Oct 30 21:41:34.363711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:34.363699    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" path="/var/lib/kubelet/pods/f060ad2c-248a-4f3a-a866-1ac2d890bbfd/volumes"
Oct 30 21:41:37.921353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.921321    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"]
Oct 30 21:41:37.922085 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922070    5786 cpu_manager.go:414] "containerMap: removing container" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy"
Oct 30 21:41:37.922128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922087    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy"
Oct 30 21:41:37.922128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922099    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy"
Oct 30 21:41:37.922128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922104    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy"
Oct 30 21:41:37.922128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922115    5786 cpu_manager.go:414] "containerMap: removing container" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy"
Oct 30 21:41:37.922128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922119    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922134    5786 cpu_manager.go:414] "containerMap: removing container" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922139    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922154    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922158    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922168    5786 cpu_manager.go:414] "containerMap: removing container" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922172    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922179    5786 cpu_manager.go:414] "containerMap: removing container" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922183    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922196    5786 cpu_manager.go:414] "containerMap: removing container" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922215    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922227    5786 cpu_manager.go:414] "containerMap: removing container" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy"
Oct 30 21:41:37.922305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922231    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922328    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a92f2bb-b279-427b-b8bc-9143dfcd6cff" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922336    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b15a6df7-ab5b-4b57-8a78-0e54a806e305" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922344    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="688684ad-2f63-44f1-9855-636878fecb22" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922351    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ec004d3-0158-47fa-87e1-b21eb07a8938" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922359    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7765c1e6-c88f-4863-89ab-a19648eed81e" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922366    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="040f7639-1a65-4901-bda0-8f7c852045a3" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922370    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922376    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="3760152e-c736-402c-89b3-b862b8fd1193" containerName="istio-proxy"
Oct 30 21:41:37.922646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.922381    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="f060ad2c-248a-4f3a-a866-1ac2d890bbfd" containerName="istio-proxy"
Oct 30 21:41:37.927699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:37.927680    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.010783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.010585    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"]
Oct 30 21:41:38.047342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.047312    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:41:38.047527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.047512    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:41:38.102360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.102316    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt24j\" (UniqueName: \"kubernetes.io/projected/03dd650a-02ff-4244-9434-e5c658f80548-kube-api-access-mt24j\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.107312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.102880    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.107312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.107080    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.208348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.208252    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.208534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.208353    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt24j\" (UniqueName: \"kubernetes.io/projected/03dd650a-02ff-4244-9434-e5c658f80548-kube-api-access-mt24j\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.208603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.208537    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.208878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.208853    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.211050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.211027    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.305844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.304949    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt24j\" (UniqueName: \"kubernetes.io/projected/03dd650a-02ff-4244-9434-e5c658f80548-kube-api-access-mt24j\") pod \"maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:38.542058 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:38.541980    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:39.210333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:39.210302    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"]
Oct 30 21:41:39.211766 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:41:39.211740    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03dd650a_02ff_4244_9434_e5c658f80548.slice/crio-b4580a2812a60cc746f463af61818cdf96d7ab6b9e0e562734e3b6a0020d81cc WatchSource:0}: Error finding container b4580a2812a60cc746f463af61818cdf96d7ab6b9e0e562734e3b6a0020d81cc: Status 404 returned error can't find the container with id b4580a2812a60cc746f463af61818cdf96d7ab6b9e0e562734e3b6a0020d81cc
Oct 30 21:41:39.904529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:39.904489    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6" event={"ID":"03dd650a-02ff-4244-9434-e5c658f80548","Type":"ContainerStarted","Data":"3062c5bc348f8375c5fb4a2457cec341690067fa6b428a168c7051fcf280f834"}
Oct 30 21:41:39.904842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:39.904821    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6" event={"ID":"03dd650a-02ff-4244-9434-e5c658f80548","Type":"ContainerStarted","Data":"b4580a2812a60cc746f463af61818cdf96d7ab6b9e0e562734e3b6a0020d81cc"}
Oct 30 21:41:40.035467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:40.035411    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6" podStartSLOduration=3.035397603 podStartE2EDuration="3.035397603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:40.032263948 +0000 UTC m=+1755.903383069" watchObservedRunningTime="2026-10-30 21:41:40.035397603 +0000 UTC m=+1755.906516721"
Oct 30 21:41:40.913500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:40.913473    5786 generic.go:356] "Generic (PLEG): container finished" podID="03dd650a-02ff-4244-9434-e5c658f80548" containerID="3062c5bc348f8375c5fb4a2457cec341690067fa6b428a168c7051fcf280f834" exitCode=0
Oct 30 21:41:40.913910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:40.913559    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6" event={"ID":"03dd650a-02ff-4244-9434-e5c658f80548","Type":"ContainerDied","Data":"3062c5bc348f8375c5fb4a2457cec341690067fa6b428a168c7051fcf280f834"}
Oct 30 21:41:42.421321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.421291    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:42.436929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.436906    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"]
Oct 30 21:41:42.437097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.437076    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy" containerID="cri-o://1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24" gracePeriod=30
Oct 30 21:41:42.582345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.580120    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03dd650a-02ff-4244-9434-e5c658f80548-kube-api-access-mt24j\" (UniqueName: \"kubernetes.io/projected/03dd650a-02ff-4244-9434-e5c658f80548-kube-api-access-mt24j\") pod \"03dd650a-02ff-4244-9434-e5c658f80548\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") "
Oct 30 21:41:42.582345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.580279    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-openshift-ca\") pod \"03dd650a-02ff-4244-9434-e5c658f80548\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") "
Oct 30 21:41:42.582345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.580476    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-xks-ca\") pod \"03dd650a-02ff-4244-9434-e5c658f80548\" (UID: \"03dd650a-02ff-4244-9434-e5c658f80548\") "
Oct 30 21:41:42.582683 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.582655    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-openshift-ca" pod "03dd650a-02ff-4244-9434-e5c658f80548" (UID: "03dd650a-02ff-4244-9434-e5c658f80548"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:42.585720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.585694    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03dd650a-02ff-4244-9434-e5c658f80548-kube-api-access-mt24j" pod "03dd650a-02ff-4244-9434-e5c658f80548" (UID: "03dd650a-02ff-4244-9434-e5c658f80548"). InnerVolumeSpecName "kube-api-access-mt24j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:42.586913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.586892    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-xks-ca" pod "03dd650a-02ff-4244-9434-e5c658f80548" (UID: "03dd650a-02ff-4244-9434-e5c658f80548"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:41:42.684607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.684565    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:42.684607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.684607    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt24j\" (UniqueName: \"kubernetes.io/projected/03dd650a-02ff-4244-9434-e5c658f80548-kube-api-access-mt24j\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:42.684872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.684624    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03dd650a-02ff-4244-9434-e5c658f80548-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:42.931345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.931314    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6" event={"ID":"03dd650a-02ff-4244-9434-e5c658f80548","Type":"ContainerDied","Data":"b4580a2812a60cc746f463af61818cdf96d7ab6b9e0e562734e3b6a0020d81cc"}
Oct 30 21:41:42.931502 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.931361    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4580a2812a60cc746f463af61818cdf96d7ab6b9e0e562734e3b6a0020d81cc"
Oct 30 21:41:42.931502 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:42.931367    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"
Oct 30 21:41:46.973064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:46.973032    5786 generic.go:356] "Generic (PLEG): container finished" podID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerID="4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761" exitCode=0
Oct 30 21:41:46.973559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:46.973103    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" event={"ID":"ed744068-8b59-4580-b1ec-e34a5f40a63e","Type":"ContainerDied","Data":"4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761"}
Oct 30 21:41:46.973559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:46.973484    5786 scope.go:122] "RemoveContainer" containerID="4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761"
Oct 30 21:41:46.974984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:46.974954    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:41:46.975072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:46.975021    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:41:46.975072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:46.975052    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:41:47.303181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.303135    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" containerID="cri-o://8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c" gracePeriod=45
Oct 30 21:41:47.615137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.615113    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:41:47.677265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.677231    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-model-cache\") pod \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") "
Oct 30 21:41:47.677476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.677393    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tmp-dir\") pod \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") "
Oct 30 21:41:47.677476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.677429    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e82f1a15-4e36-4179-8e3b-653dbe03b86b-kube-api-access-xqv6p\" (UniqueName: \"kubernetes.io/projected/e82f1a15-4e36-4179-8e3b-653dbe03b86b-kube-api-access-xqv6p\") pod \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") "
Oct 30 21:41:47.677476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.677468    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-model-cache" pod "e82f1a15-4e36-4179-8e3b-653dbe03b86b" (UID: "e82f1a15-4e36-4179-8e3b-653dbe03b86b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.677671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.677506    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-home\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-home\") pod \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") "
Oct 30 21:41:47.677671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.677639    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tmp-dir" pod "e82f1a15-4e36-4179-8e3b-653dbe03b86b" (UID: "e82f1a15-4e36-4179-8e3b-653dbe03b86b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.677768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.677730    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-dshm\") pod \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") "
Oct 30 21:41:47.677824 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.677815    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tls-certs\" (UniqueName: \"kubernetes.io/secret/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tls-certs\") pod \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\" (UID: \"e82f1a15-4e36-4179-8e3b-653dbe03b86b\") "
Oct 30 21:41:47.678102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.678078    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-home" pod "e82f1a15-4e36-4179-8e3b-653dbe03b86b" (UID: "e82f1a15-4e36-4179-8e3b-653dbe03b86b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.678609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.678584    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.678803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.678613    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.678803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.678628    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.686027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.685999    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tls-certs" pod "e82f1a15-4e36-4179-8e3b-653dbe03b86b" (UID: "e82f1a15-4e36-4179-8e3b-653dbe03b86b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:41:47.686838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.686803    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-dshm" pod "e82f1a15-4e36-4179-8e3b-653dbe03b86b" (UID: "e82f1a15-4e36-4179-8e3b-653dbe03b86b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.688632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.688607    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e82f1a15-4e36-4179-8e3b-653dbe03b86b-kube-api-access-xqv6p" pod "e82f1a15-4e36-4179-8e3b-653dbe03b86b" (UID: "e82f1a15-4e36-4179-8e3b-653dbe03b86b"). InnerVolumeSpecName "kube-api-access-xqv6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:47.719069 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.719049    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:41:47.783631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.783541    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-podinfo\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.783808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.783695    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-socket\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.783808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.783756    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-token\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-token\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.783921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.783859    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0910c2f7-ba5e-48fe-baaa-087201160f94-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0910c2f7-ba5e-48fe-baaa-087201160f94-istiod-ca-cert\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.783974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.783925    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-credential-socket\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.788095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.784047    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-kube-api-access-sq5rb\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-kube-api-access-sq5rb\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.788095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.784128    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-data\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.788095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.784220    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-envoy\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.788095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.784285    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-certs\") pod \"0910c2f7-ba5e-48fe-baaa-087201160f94\" (UID: \"0910c2f7-ba5e-48fe-baaa-087201160f94\") "
Oct 30 21:41:47.788095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.785037    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqv6p\" (UniqueName: \"kubernetes.io/projected/e82f1a15-4e36-4179-8e3b-653dbe03b86b-kube-api-access-xqv6p\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.788095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.785062    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e82f1a15-4e36-4179-8e3b-653dbe03b86b-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.788095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.785077    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e82f1a15-4e36-4179-8e3b-653dbe03b86b-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.796037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.795583    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-credential-socket" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.796037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.795799    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-data" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.796037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.796005    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0910c2f7-ba5e-48fe-baaa-087201160f94-istiod-ca-cert" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:47.798403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.798333    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-socket" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.800513 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.800396    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-podinfo" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:47.800513 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.800471    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-envoy" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.800685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.800620    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-kube-api-access-sq5rb" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "kube-api-access-sq5rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:47.801577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.801542    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-token" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:47.805320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.805287    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-certs" pod "0910c2f7-ba5e-48fe-baaa-087201160f94" (UID: "0910c2f7-ba5e-48fe-baaa-087201160f94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.887946    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.887984    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.887997    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0910c2f7-ba5e-48fe-baaa-087201160f94-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.888013    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.888026    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq5rb\" (UniqueName: \"kubernetes.io/projected/0910c2f7-ba5e-48fe-baaa-087201160f94-kube-api-access-sq5rb\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.888039    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.888051    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.888063    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0910c2f7-ba5e-48fe-baaa-087201160f94-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:47.888128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:47.888074    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0910c2f7-ba5e-48fe-baaa-087201160f94-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:48.019797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.019762    5786 generic.go:356] "Generic (PLEG): container finished" podID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerID="1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24" exitCode=0
Oct 30 21:41:48.020265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.019834    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"
Oct 30 21:41:48.020265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.019839    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" event={"ID":"0910c2f7-ba5e-48fe-baaa-087201160f94","Type":"ContainerDied","Data":"1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24"}
Oct 30 21:41:48.020265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.019915    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl" event={"ID":"0910c2f7-ba5e-48fe-baaa-087201160f94","Type":"ContainerDied","Data":"2ec17d0e698cd1662a3e20accfcc25d5f9796d0411043009080ce6e34841c090"}
Oct 30 21:41:48.020265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.019939    5786 scope.go:122] "RemoveContainer" containerID="1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24"
Oct 30 21:41:48.022146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.022126    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" event={"ID":"ed744068-8b59-4580-b1ec-e34a5f40a63e","Type":"ContainerStarted","Data":"6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c"}
Oct 30 21:41:48.024431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.024412    5786 generic.go:356] "Generic (PLEG): container finished" podID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerID="8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c" exitCode=0
Oct 30 21:41:48.024653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.024462    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"
Oct 30 21:41:48.024734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.024476    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" event={"ID":"e82f1a15-4e36-4179-8e3b-653dbe03b86b","Type":"ContainerDied","Data":"8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c"}
Oct 30 21:41:48.024799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.024764    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" event={"ID":"e82f1a15-4e36-4179-8e3b-653dbe03b86b","Type":"ContainerDied","Data":"4cf76b0d691cef6a5f7a42a8467bdb7b43e8a4aeb50a4fe5a8003c088e506143"}
Oct 30 21:41:48.029150 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.029127    5786 generic.go:356] "Generic (PLEG): container finished" podID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerID="5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23" exitCode=0
Oct 30 21:41:48.029278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.029168    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" event={"ID":"69bebe1c-95dc-4834-b836-bbd28f21f95d","Type":"ContainerDied","Data":"5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23"}
Oct 30 21:41:48.029629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.029601    5786 scope.go:122] "RemoveContainer" containerID="5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23"
Oct 30 21:41:48.032379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.032347    5786 scope.go:122] "RemoveContainer" containerID="1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24"
Oct 30 21:41:48.032866 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:48.032841    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24\": container with ID starting with 1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24 not found: ID does not exist" containerID="1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24"
Oct 30 21:41:48.032963 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.032874    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24"} err="failed to get container status \"1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24\": rpc error: code = NotFound desc = could not find container \"1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24\": container with ID starting with 1867206b8578799ce18b8e5853159552f055e11892317ca83e2839245a722b24 not found: ID does not exist"
Oct 30 21:41:48.032963 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.032896    5786 scope.go:122] "RemoveContainer" containerID="8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c"
Oct 30 21:41:48.033139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.033046    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:41:48.033339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.033137    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:41:48.033339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.033188    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:41:48.046575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.046557    5786 scope.go:122] "RemoveContainer" containerID="8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c"
Oct 30 21:41:48.046818 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:48.046803    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c\": container with ID starting with 8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c not found: ID does not exist" containerID="8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c"
Oct 30 21:41:48.046866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.046824    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c"} err="failed to get container status \"8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c\": rpc error: code = NotFound desc = could not find container \"8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c\": container with ID starting with 8fca89b7b80b8db7a8d8cb8dff066093ecc54902f6e4d45d0d4243eca2caaa3c not found: ID does not exist"
Oct 30 21:41:48.345753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.345720    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"]
Oct 30 21:41:48.350534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.350512    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-s6fhl"]
Oct 30 21:41:48.390179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.390146    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" path="/var/lib/kubelet/pods/0910c2f7-ba5e-48fe-baaa-087201160f94/volumes"
Oct 30 21:41:48.481232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.480044    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"]
Oct 30 21:41:48.485161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.485134    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht"]
Oct 30 21:41:48.498736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.498699    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9ac5c5/simulator-w6-9ac5c5-kserve-7866684f8c-dmtht" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:48.668291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.667602    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:41:48.673231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.671309    5786 patch_prober.go:29] interesting pod/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 21:41:48.673231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:48.671363    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 21:41:49.039197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:49.039119    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" event={"ID":"69bebe1c-95dc-4834-b836-bbd28f21f95d","Type":"ContainerStarted","Data":"2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197"}
Oct 30 21:41:49.580233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:49.575398    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:41:49.603403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:49.602642    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"]
Oct 30 21:41:49.687240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:49.686369    5786 patch_prober.go:29] interesting pod/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 21:41:49.687240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:49.686440    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 21:41:49.701391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:49.701352    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"]
Oct 30 21:41:50.083940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:50.080781    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:41:50.273338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:50.230597    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:41:50.273338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:50.230660    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 21:41:50.414020 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:50.413982    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" path="/var/lib/kubelet/pods/e82f1a15-4e36-4179-8e3b-653dbe03b86b/volumes"
Oct 30 21:41:50.674288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:50.673652    5786 patch_prober.go:29] interesting pod/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 21:41:50.674288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:50.673708    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 21:41:51.016515 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:51.016436    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:41:51.016515 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:51.016493    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 21:41:51.675229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:51.675146    5786 patch_prober.go:29] interesting pod/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 21:41:51.675746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:51.675253    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 21:41:52.018093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:52.017584    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:41:52.018093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:52.017656    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 21:41:52.420436 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:41:52.420400    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:41:52.669618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:52.669571    5786 patch_prober.go:29] interesting pod/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 21:41:52.669928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:52.669895    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 21:41:53.019639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:53.019599    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:41:53.020285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:53.020253    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 21:41:53.685274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:53.684555    5786 patch_prober.go:29] interesting pod/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 21:41:53.685274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:53.684613    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 21:41:54.015902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:54.015824    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:41:54.015902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:54.015872    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 21:41:54.603074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:54.603032    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk" podUID="aead0097-88ae-4970-81c2-f856c8000169" containerName="payload-processing" containerID="cri-o://4882cb1318d13148f4564a602ad4cee02021337a646f2b5d3050a075a1c58620" gracePeriod=25
Oct 30 21:41:54.668529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:54.668498    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:41:54.669771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:54.669748    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:41:54.702842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:54.702807    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c" podUID="e244748d-aecf-4ab3-973f-d0eae1307c1c" containerName="payload-pre-processing" containerID="cri-o://8b2d7c23c4d6636069373318984d16728594fe35bb79b033ceea3174a590ca6d" gracePeriod=25
Oct 30 21:41:55.020806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:55.020780    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:41:55.021234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:55.021201    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:41:55.021893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:55.021877    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:41:55.731671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:55.731635    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk" podUID="aead0097-88ae-4970-81c2-f856c8000169" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:41:55.732051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:41:55.731676    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c" podUID="e244748d-aecf-4ab3-973f-d0eae1307c1c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:42:05.731937 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:05.731893    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c" podUID="e244748d-aecf-4ab3-973f-d0eae1307c1c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:42:05.732403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:05.731942    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk" podUID="aead0097-88ae-4970-81c2-f856c8000169" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:42:07.369063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:07.369027    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"]
Oct 30 21:42:07.369594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:07.369251    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy" containerID="cri-o://7ecf255d8f1f506c4ef53848330dc1276635e16e083e2128b4ee51322afd2299" gracePeriod=30
Oct 30 21:42:07.418784 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:07.418757    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"]
Oct 30 21:42:07.418956 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:07.418917    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy" containerID="cri-o://b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c" gracePeriod=30
Oct 30 21:42:07.422140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:07.422119    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"]
Oct 30 21:42:07.422298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:07.422278    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy" containerID="cri-o://70eba89230707eb27224e7e2e1c724702716f761af70a2a927b236a4c395440b" gracePeriod=30
Oct 30 21:42:07.467892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:07.467862    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"]
Oct 30 21:42:07.468164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:07.468068    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy" containerID="cri-o://c5caabc703a8a4260582d14f092027e58e388fcb8931f9c4d81c229799aebcec" gracePeriod=30
Oct 30 21:42:12.624444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.623578    5786 patch_prober.go:29] interesting pod/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:42:12.624444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.623633    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:42:12.787550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.787487    5786 generic.go:356] "Generic (PLEG): container finished" podID="18da9832-9fe6-4381-96a2-539560c6663b" containerID="7ecf255d8f1f506c4ef53848330dc1276635e16e083e2128b4ee51322afd2299" exitCode=0
Oct 30 21:42:12.787727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.787711    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" event={"ID":"18da9832-9fe6-4381-96a2-539560c6663b","Type":"ContainerDied","Data":"7ecf255d8f1f506c4ef53848330dc1276635e16e083e2128b4ee51322afd2299"}
Oct 30 21:42:12.790535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.790442    5786 generic.go:356] "Generic (PLEG): container finished" podID="b6f2e61a-2162-4698-acea-b14639ddff20" containerID="70eba89230707eb27224e7e2e1c724702716f761af70a2a927b236a4c395440b" exitCode=0
Oct 30 21:42:12.790535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.790513    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" event={"ID":"b6f2e61a-2162-4698-acea-b14639ddff20","Type":"ContainerDied","Data":"70eba89230707eb27224e7e2e1c724702716f761af70a2a927b236a4c395440b"}
Oct 30 21:42:12.792865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.792842    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:42:12.794892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.794168    5786 generic.go:356] "Generic (PLEG): container finished" podID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerID="c5caabc703a8a4260582d14f092027e58e388fcb8931f9c4d81c229799aebcec" exitCode=0
Oct 30 21:42:12.794892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.794292    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" event={"ID":"1f382dea-dd70-4ebc-accb-a9bab6fc577f","Type":"ContainerDied","Data":"c5caabc703a8a4260582d14f092027e58e388fcb8931f9c4d81c229799aebcec"}
Oct 30 21:42:12.794892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.794336    5786 scope.go:122] "RemoveContainer" containerID="e105ef17d6ae46cb10254c555c781b7a4472cd0809c1d506dc637237b0d507f8"
Oct 30 21:42:12.805731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.805711    5786 generic.go:356] "Generic (PLEG): container finished" podID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerID="b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c" exitCode=0
Oct 30 21:42:12.805844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.805744    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" event={"ID":"99f3b286-fd29-4721-92c1-3d65435a48d4","Type":"ContainerDied","Data":"b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c"}
Oct 30 21:42:12.805844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.805782    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f" event={"ID":"99f3b286-fd29-4721-92c1-3d65435a48d4","Type":"ContainerDied","Data":"0c9f2843b67255b34182f7ec0f9ed1c370d8ebef095ac89b988ed517a7129b35"}
Oct 30 21:42:12.805968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.805856    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"
Oct 30 21:42:12.807565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.807503    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:42:12.815788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.815771    5786 scope.go:122] "RemoveContainer" containerID="b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c"
Oct 30 21:42:12.825800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.825784    5786 scope.go:122] "RemoveContainer" containerID="b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c"
Oct 30 21:42:12.826100 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:42:12.826051    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c\": container with ID starting with b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c not found: ID does not exist" containerID="b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c"
Oct 30 21:42:12.826169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.826111    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c"} err="failed to get container status \"b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c\": rpc error: code = NotFound desc = could not find container \"b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c\": container with ID starting with b6f36047eddff7f37498b68fc4d34b2639ce823f3b126322c93d3266ef85088c not found: ID does not exist"
Oct 30 21:42:12.888053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.887990    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:42:12.936990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.936958    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-socket\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.937187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937025    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-podinfo\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.937187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937069    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-envoy\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.937187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937100    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6f2e61a-2162-4698-acea-b14639ddff20-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6f2e61a-2162-4698-acea-b14639ddff20-istiod-ca-cert\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937138    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-data\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937273    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-envoy\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937274    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-socket" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.937404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937309    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-certs\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.937404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937369    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-token\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-token\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.937533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937409    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-certs\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937445    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99f3b286-fd29-4721-92c1-3d65435a48d4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f3b286-fd29-4721-92c1-3d65435a48d4-istiod-ca-cert\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.937599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937577    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b6f2e61a-2162-4698-acea-b14639ddff20-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6f2e61a-2162-4698-acea-b14639ddff20-istio-podinfo\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937640    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-socket\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937683    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-istio-token\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-istio-token\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937729    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-data" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.937974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937743    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-kube-api-access-57qpc\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-kube-api-access-57qpc\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937872    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-credential-socket\") pod \"b6f2e61a-2162-4698-acea-b14639ddff20\" (UID: \"b6f2e61a-2162-4698-acea-b14639ddff20\") "
Oct 30 21:42:12.937974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937925    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-credential-socket\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.937974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937951    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-data\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.938427 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.938012    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-kube-api-access-qlmmq\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-kube-api-access-qlmmq\") pod \"99f3b286-fd29-4721-92c1-3d65435a48d4\" (UID: \"99f3b286-fd29-4721-92c1-3d65435a48d4\") "
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.937945    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f2e61a-2162-4698-acea-b14639ddff20-istiod-ca-cert" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.938234    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-certs" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.938274    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-certs" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.938573    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f3b286-fd29-4721-92c1-3d65435a48d4-istiod-ca-cert" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.938594    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-credential-socket" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.938721    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-credential-socket" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939004    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-data" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939073    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-socket" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939506    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939528    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939544    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939559    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939573    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6f2e61a-2162-4698-acea-b14639ddff20-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939589    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939603    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939618    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939633    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f3b286-fd29-4721-92c1-3d65435a48d4-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.939687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.939647    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:12.941171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.941149    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-envoy" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.941288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.941253    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-kube-api-access-qlmmq" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "kube-api-access-qlmmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:12.941404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.941289    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-kube-api-access-57qpc" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "kube-api-access-57qpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:12.941404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.941307    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-token" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:12.941404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.941311    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-envoy" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:12.942471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.942404    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6f2e61a-2162-4698-acea-b14639ddff20-istio-podinfo" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:42:12.942471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.942439    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-podinfo" pod "99f3b286-fd29-4721-92c1-3d65435a48d4" (UID: "99f3b286-fd29-4721-92c1-3d65435a48d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:42:12.942889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:12.942861    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-istio-token" pod "b6f2e61a-2162-4698-acea-b14639ddff20" (UID: "b6f2e61a-2162-4698-acea-b14639ddff20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.041839    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-data\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.041934    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-token\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-token\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.041967    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-podinfo\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.041993    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-certs\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.042077    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-credential-socket\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.042195    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istiod-ca-cert\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.042322    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-socket\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.042400    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-envoy\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043038    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-kube-api-access-rf59d\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-kube-api-access-rf59d\") pod \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\" (UID: \"1f382dea-dd70-4ebc-accb-a9bab6fc577f\") "
Oct 30 21:42:13.043857 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043802    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6f2e61a-2162-4698-acea-b14639ddff20-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.043914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043856    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.043914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043872    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6f2e61a-2162-4698-acea-b14639ddff20-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.043914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043886    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.043914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043900    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57qpc\" (UniqueName: \"kubernetes.io/projected/b6f2e61a-2162-4698-acea-b14639ddff20-kube-api-access-57qpc\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.043914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043913    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlmmq\" (UniqueName: \"kubernetes.io/projected/99f3b286-fd29-4721-92c1-3d65435a48d4-kube-api-access-qlmmq\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.044166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043926    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.044166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.043939    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f3b286-fd29-4721-92c1-3d65435a48d4-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.044308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.044289    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-data" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.044632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.044609    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-credential-socket" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.045556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.045065    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istiod-ca-cert" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:13.045697 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.045337    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-socket" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.045788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.045510    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-certs" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.047968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.047948    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-envoy" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.048222 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.048160    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-kube-api-access-rf59d" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "kube-api-access-rf59d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:13.048926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.048900    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-token" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:13.049017 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.048930    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-podinfo" pod "1f382dea-dd70-4ebc-accb-a9bab6fc577f" (UID: "1f382dea-dd70-4ebc-accb-a9bab6fc577f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:42:13.105341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.105315    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:42:13.145434 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145408    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.145434 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145433    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf59d\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-kube-api-access-rf59d\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.145665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145444    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.145665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145453    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.145665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145461    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.145665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145469    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.145665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145477    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.145665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145485    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f382dea-dd70-4ebc-accb-a9bab6fc577f-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.145665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.145493    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f382dea-dd70-4ebc-accb-a9bab6fc577f-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.246597    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-socket\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.246665    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/18da9832-9fe6-4381-96a2-539560c6663b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18da9832-9fe6-4381-96a2-539560c6663b-istio-podinfo\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.247109    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-socket" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.247137    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-envoy\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.247719    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-data\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.247850    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-certs\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.247949    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18da9832-9fe6-4381-96a2-539560c6663b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18da9832-9fe6-4381-96a2-539560c6663b-istiod-ca-cert\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.248006    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-istio-token\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-istio-token\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.248037    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-credential-socket\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.276229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.248120    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-kube-api-access-xfxtz\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-kube-api-access-xfxtz\") pod \"18da9832-9fe6-4381-96a2-539560c6663b\" (UID: \"18da9832-9fe6-4381-96a2-539560c6663b\") "
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.279472    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-data" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.279623    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-certs" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.279846    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18da9832-9fe6-4381-96a2-539560c6663b-istiod-ca-cert" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.280526    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-credential-socket" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.281856    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.281879    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.281893    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18da9832-9fe6-4381-96a2-539560c6663b-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.281907    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.285230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.281921    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.288238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.287936    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-kube-api-access-xfxtz" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "kube-api-access-xfxtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:13.288238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.288006    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-istio-token" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:13.288956 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.288671    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-envoy" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:13.291265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.291242    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18da9832-9fe6-4381-96a2-539560c6663b-istio-podinfo" pod "18da9832-9fe6-4381-96a2-539560c6663b" (UID: "18da9832-9fe6-4381-96a2-539560c6663b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:42:13.390087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.389926    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18da9832-9fe6-4381-96a2-539560c6663b-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.390087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.389956    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18da9832-9fe6-4381-96a2-539560c6663b-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.390087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.389970    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.390087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.389985    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfxtz\" (UniqueName: \"kubernetes.io/projected/18da9832-9fe6-4381-96a2-539560c6663b-kube-api-access-xfxtz\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:13.409817 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.409793    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"]
Oct 30 21:42:13.427381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.427357    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-4gc9f"]
Oct 30 21:42:13.814180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.814148    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"
Oct 30 21:42:13.814180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.814168    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8" event={"ID":"18da9832-9fe6-4381-96a2-539560c6663b","Type":"ContainerDied","Data":"981deafda8c7fe66ce8507e4dc5ff11ad14805c048e0bb426fc8b5725f08a284"}
Oct 30 21:42:13.814770 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.814286    5786 scope.go:122] "RemoveContainer" containerID="7ecf255d8f1f506c4ef53848330dc1276635e16e083e2128b4ee51322afd2299"
Oct 30 21:42:13.816133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.816115    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"
Oct 30 21:42:13.816133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.816120    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn" event={"ID":"b6f2e61a-2162-4698-acea-b14639ddff20","Type":"ContainerDied","Data":"5902f55d19c3f206a61933f5a33966ec2529a41978bd8f82f671c48fde6a676f"}
Oct 30 21:42:13.817921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.817899    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"
Oct 30 21:42:13.818050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.817898    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7" event={"ID":"1f382dea-dd70-4ebc-accb-a9bab6fc577f","Type":"ContainerDied","Data":"08549e1a943aba771f18544784e93e2024364e85db5c700b4bc3d7303758931f"}
Oct 30 21:42:13.826553 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.826534    5786 scope.go:122] "RemoveContainer" containerID="70eba89230707eb27224e7e2e1c724702716f761af70a2a927b236a4c395440b"
Oct 30 21:42:13.837683 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.837666    5786 scope.go:122] "RemoveContainer" containerID="c5caabc703a8a4260582d14f092027e58e388fcb8931f9c4d81c229799aebcec"
Oct 30 21:42:13.984089 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.984057    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"]
Oct 30 21:42:13.988026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:13.988002    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-dvvz8"]
Oct 30 21:42:14.095573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.095495    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"]
Oct 30 21:42:14.098520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.098497    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-6plv7"]
Oct 30 21:42:14.206873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.206836    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"]
Oct 30 21:42:14.210226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.210185    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9ac5c5-openshift-default-5f9f955bc7-plsvn"]
Oct 30 21:42:14.383844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.383767    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18da9832-9fe6-4381-96a2-539560c6663b" path="/var/lib/kubelet/pods/18da9832-9fe6-4381-96a2-539560c6663b/volumes"
Oct 30 21:42:14.384248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.384234    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" path="/var/lib/kubelet/pods/1f382dea-dd70-4ebc-accb-a9bab6fc577f/volumes"
Oct 30 21:42:14.384680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.384668    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" path="/var/lib/kubelet/pods/99f3b286-fd29-4721-92c1-3d65435a48d4/volumes"
Oct 30 21:42:14.385426 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.385412    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" path="/var/lib/kubelet/pods/b6f2e61a-2162-4698-acea-b14639ddff20/volumes"
Oct 30 21:42:14.476634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.476589    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"]
Oct 30 21:42:14.477332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477316    5786 cpu_manager.go:414] "containerMap: removing container" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy"
Oct 30 21:42:14.477399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477333    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy"
Oct 30 21:42:14.477399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477345    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main"
Oct 30 21:42:14.477399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477353    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main"
Oct 30 21:42:14.477399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477370    5786 cpu_manager.go:414] "containerMap: removing container" podUID="03dd650a-02ff-4244-9434-e5c658f80548" containerName="revoke-keys"
Oct 30 21:42:14.477399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477377    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03dd650a-02ff-4244-9434-e5c658f80548" containerName="revoke-keys"
Oct 30 21:42:14.477399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477391    5786 cpu_manager.go:414] "containerMap: removing container" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy"
Oct 30 21:42:14.477399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477395    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477411    5786 cpu_manager.go:414] "containerMap: removing container" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477415    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477422    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477428    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477438    5786 cpu_manager.go:414] "containerMap: removing container" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477445    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477581    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="18da9832-9fe6-4381-96a2-539560c6663b" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477591    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6f2e61a-2162-4698-acea-b14639ddff20" containerName="istio-proxy"
Oct 30 21:42:14.477607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477601    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="0910c2f7-ba5e-48fe-baaa-087201160f94" containerName="istio-proxy"
Oct 30 21:42:14.477868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477623    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="99f3b286-fd29-4721-92c1-3d65435a48d4" containerName="istio-proxy"
Oct 30 21:42:14.477868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477631    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e82f1a15-4e36-4179-8e3b-653dbe03b86b" containerName="main"
Oct 30 21:42:14.477868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477637    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="03dd650a-02ff-4244-9434-e5c658f80548" containerName="revoke-keys"
Oct 30 21:42:14.477868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477642    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy"
Oct 30 21:42:14.477868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.477648    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy"
Oct 30 21:42:14.482823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.482808    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.508094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.508075    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.508271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.508251    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tlz5\" (UniqueName: \"kubernetes.io/projected/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-kube-api-access-5tlz5\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.508335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.508300    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.517449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.517425    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"]
Oct 30 21:42:14.534217 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.534178    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:42:14.534347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.534310    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:42:14.610274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.610243    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tlz5\" (UniqueName: \"kubernetes.io/projected/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-kube-api-access-5tlz5\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.610396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.610335    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.610579 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.610565    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.611061 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.611038    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.613586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.613565    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.676812 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.676764    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tlz5\" (UniqueName: \"kubernetes.io/projected/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-kube-api-access-5tlz5\") pod \"maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.795991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.795964    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:14.835296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.834648    5786 generic.go:356] "Generic (PLEG): container finished" podID="e244748d-aecf-4ab3-973f-d0eae1307c1c" containerID="8b2d7c23c4d6636069373318984d16728594fe35bb79b033ceea3174a590ca6d" exitCode=0
Oct 30 21:42:14.835296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.834796    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c" event={"ID":"e244748d-aecf-4ab3-973f-d0eae1307c1c","Type":"ContainerDied","Data":"8b2d7c23c4d6636069373318984d16728594fe35bb79b033ceea3174a590ca6d"}
Oct 30 21:42:14.839364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.839009    5786 generic.go:356] "Generic (PLEG): container finished" podID="aead0097-88ae-4970-81c2-f856c8000169" containerID="4882cb1318d13148f4564a602ad4cee02021337a646f2b5d3050a075a1c58620" exitCode=0
Oct 30 21:42:14.839364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.839299    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk" event={"ID":"aead0097-88ae-4970-81c2-f856c8000169","Type":"ContainerDied","Data":"4882cb1318d13148f4564a602ad4cee02021337a646f2b5d3050a075a1c58620"}
Oct 30 21:42:14.936860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:14.936827    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:42:15.017027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.016993    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aead0097-88ae-4970-81c2-f856c8000169-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aead0097-88ae-4970-81c2-f856c8000169-plugins-config-volume\") pod \"aead0097-88ae-4970-81c2-f856c8000169\" (UID: \"aead0097-88ae-4970-81c2-f856c8000169\") "
Oct 30 21:42:15.017260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.017195    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aead0097-88ae-4970-81c2-f856c8000169-kube-api-access-rx69q\" (UniqueName: \"kubernetes.io/projected/aead0097-88ae-4970-81c2-f856c8000169-kube-api-access-rx69q\") pod \"aead0097-88ae-4970-81c2-f856c8000169\" (UID: \"aead0097-88ae-4970-81c2-f856c8000169\") "
Oct 30 21:42:15.018499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.018474    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aead0097-88ae-4970-81c2-f856c8000169-plugins-config-volume" pod "aead0097-88ae-4970-81c2-f856c8000169" (UID: "aead0097-88ae-4970-81c2-f856c8000169"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:15.020941 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.020897    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aead0097-88ae-4970-81c2-f856c8000169-kube-api-access-rx69q" pod "aead0097-88ae-4970-81c2-f856c8000169" (UID: "aead0097-88ae-4970-81c2-f856c8000169"). InnerVolumeSpecName "kube-api-access-rx69q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:15.102582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.102554    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:42:15.118948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.118918    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e244748d-aecf-4ab3-973f-d0eae1307c1c-kube-api-access-hfcc7\" (UniqueName: \"kubernetes.io/projected/e244748d-aecf-4ab3-973f-d0eae1307c1c-kube-api-access-hfcc7\") pod \"e244748d-aecf-4ab3-973f-d0eae1307c1c\" (UID: \"e244748d-aecf-4ab3-973f-d0eae1307c1c\") "
Oct 30 21:42:15.119081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.118955    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e244748d-aecf-4ab3-973f-d0eae1307c1c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e244748d-aecf-4ab3-973f-d0eae1307c1c-plugins-config-volume\") pod \"e244748d-aecf-4ab3-973f-d0eae1307c1c\" (UID: \"e244748d-aecf-4ab3-973f-d0eae1307c1c\") "
Oct 30 21:42:15.119537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.119515    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx69q\" (UniqueName: \"kubernetes.io/projected/aead0097-88ae-4970-81c2-f856c8000169-kube-api-access-rx69q\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:15.119537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.119538    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aead0097-88ae-4970-81c2-f856c8000169-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:15.119658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.119539    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e244748d-aecf-4ab3-973f-d0eae1307c1c-plugins-config-volume" pod "e244748d-aecf-4ab3-973f-d0eae1307c1c" (UID: "e244748d-aecf-4ab3-973f-d0eae1307c1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:15.121239 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.121198    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e244748d-aecf-4ab3-973f-d0eae1307c1c-kube-api-access-hfcc7" pod "e244748d-aecf-4ab3-973f-d0eae1307c1c" (UID: "e244748d-aecf-4ab3-973f-d0eae1307c1c"). InnerVolumeSpecName "kube-api-access-hfcc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:15.220622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.220596    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfcc7\" (UniqueName: \"kubernetes.io/projected/e244748d-aecf-4ab3-973f-d0eae1307c1c-kube-api-access-hfcc7\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:15.220622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.220621    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e244748d-aecf-4ab3-973f-d0eae1307c1c-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:15.236668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.236645    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"]
Oct 30 21:42:15.867588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.867513    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"
Oct 30 21:42:15.867984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.867512    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk" event={"ID":"aead0097-88ae-4970-81c2-f856c8000169","Type":"ContainerDied","Data":"dd8df5f11d05a4f426b517a2f4929bb60be8d65af0282703f8a8445fc998b914"}
Oct 30 21:42:15.868120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.868106    5786 scope.go:122] "RemoveContainer" containerID="4882cb1318d13148f4564a602ad4cee02021337a646f2b5d3050a075a1c58620"
Oct 30 21:42:15.870059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.870027    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs" event={"ID":"7128c5ce-3dcc-4743-ad9b-f63b53a2b523","Type":"ContainerStarted","Data":"e7fcc7f3eed84a8247e62fbcc8fdea95782fd8cadcf305e51e138c8e4d68198e"}
Oct 30 21:42:15.870169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.870079    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs" event={"ID":"7128c5ce-3dcc-4743-ad9b-f63b53a2b523","Type":"ContainerStarted","Data":"af4dd3499674b6116c623fff3781bb1c6a915379c0d14b2768cdcdd5d478f791"}
Oct 30 21:42:15.872251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.871848    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c" event={"ID":"e244748d-aecf-4ab3-973f-d0eae1307c1c","Type":"ContainerDied","Data":"e394752cec240f812ee94654c214c2dca0b50a921546ce6aaa206f157245d4a0"}
Oct 30 21:42:15.872251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.871855    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"
Oct 30 21:42:15.880598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:15.880581    5786 scope.go:122] "RemoveContainer" containerID="8b2d7c23c4d6636069373318984d16728594fe35bb79b033ceea3174a590ca6d"
Oct 30 21:42:16.030629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.030602    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"]
Oct 30 21:42:16.040094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.040067    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9ac5c5-78c47fc8c5-qq49c"]
Oct 30 21:42:16.143851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.143781    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"]
Oct 30 21:42:16.151071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.151044    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9ac5c5-768676c487-dqvjk"]
Oct 30 21:42:16.266723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.266681    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs" podStartSLOduration=2.266668973 podStartE2EDuration="2.266668973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:16.266383847 +0000 UTC m=+1792.137502967" watchObservedRunningTime="2026-10-30 21:42:16.266668973 +0000 UTC m=+1792.137788090"
Oct 30 21:42:16.362574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.362544    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aead0097-88ae-4970-81c2-f856c8000169" path="/var/lib/kubelet/pods/aead0097-88ae-4970-81c2-f856c8000169/volumes"
Oct 30 21:42:16.362957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.362943    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e244748d-aecf-4ab3-973f-d0eae1307c1c" path="/var/lib/kubelet/pods/e244748d-aecf-4ab3-973f-d0eae1307c1c/volumes"
Oct 30 21:42:16.883609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.883575    5786 generic.go:356] "Generic (PLEG): container finished" podID="7128c5ce-3dcc-4743-ad9b-f63b53a2b523" containerID="e7fcc7f3eed84a8247e62fbcc8fdea95782fd8cadcf305e51e138c8e4d68198e" exitCode=0
Oct 30 21:42:16.884068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:16.883625    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs" event={"ID":"7128c5ce-3dcc-4743-ad9b-f63b53a2b523","Type":"ContainerDied","Data":"e7fcc7f3eed84a8247e62fbcc8fdea95782fd8cadcf305e51e138c8e4d68198e"}
Oct 30 21:42:18.057052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.057024    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:18.163651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.163630    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-xks-ca\") pod \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") "
Oct 30 21:42:18.163795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.163702    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-openshift-ca\") pod \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") "
Oct 30 21:42:18.163795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.163758    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-kube-api-access-5tlz5\" (UniqueName: \"kubernetes.io/projected/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-kube-api-access-5tlz5\") pod \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\" (UID: \"7128c5ce-3dcc-4743-ad9b-f63b53a2b523\") "
Oct 30 21:42:18.164098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.164068    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-openshift-ca" pod "7128c5ce-3dcc-4743-ad9b-f63b53a2b523" (UID: "7128c5ce-3dcc-4743-ad9b-f63b53a2b523"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:18.164359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.164342    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:18.166022 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.165992    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-xks-ca" pod "7128c5ce-3dcc-4743-ad9b-f63b53a2b523" (UID: "7128c5ce-3dcc-4743-ad9b-f63b53a2b523"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:42:18.166405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.166379    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-kube-api-access-5tlz5" pod "7128c5ce-3dcc-4743-ad9b-f63b53a2b523" (UID: "7128c5ce-3dcc-4743-ad9b-f63b53a2b523"). InnerVolumeSpecName "kube-api-access-5tlz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:18.265811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.265777    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tlz5\" (UniqueName: \"kubernetes.io/projected/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-kube-api-access-5tlz5\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:18.265811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.265813    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7128c5ce-3dcc-4743-ad9b-f63b53a2b523-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:18.902779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.902740    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"
Oct 30 21:42:18.902779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.902757    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs" event={"ID":"7128c5ce-3dcc-4743-ad9b-f63b53a2b523","Type":"ContainerDied","Data":"af4dd3499674b6116c623fff3781bb1c6a915379c0d14b2768cdcdd5d478f791"}
Oct 30 21:42:18.902961 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:18.902828    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af4dd3499674b6116c623fff3781bb1c6a915379c0d14b2768cdcdd5d478f791"
Oct 30 21:42:19.008836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:19.008809    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"]
Oct 30 21:42:19.012742 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:19.012715    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"]
Oct 30 21:42:19.912850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:19.912817    5786 generic.go:356] "Generic (PLEG): container finished" podID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerID="862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033" exitCode=0
Oct 30 21:42:19.913386 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:19.912888    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" event={"ID":"30a4fbd5-4b68-4316-8e2b-c693757f3009","Type":"ContainerDied","Data":"862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033"}
Oct 30 21:42:19.913447 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:19.913396    5786 scope.go:122] "RemoveContainer" containerID="862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033"
Oct 30 21:42:19.915026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:19.914993    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:42:19.915124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:19.915063    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:42:19.915124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:19.915097    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:42:20.933927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:20.933884    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" event={"ID":"30a4fbd5-4b68-4316-8e2b-c693757f3009","Type":"ContainerStarted","Data":"27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f"}
Oct 30 21:42:21.709004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:21.708972    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:42:21.710301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:21.710266    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 21:42:21.710430 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:21.710332    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 21:42:22.382782 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:42:22.382745    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:42:22.610896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:22.610861    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:42:22.709484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:22.709452    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 21:42:22.709666 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:22.709503    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 21:42:23.709726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:23.709694    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 21:42:23.710086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:23.709754    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 21:42:24.708895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:24.708854    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 21:42:24.709064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:24.708900    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 21:42:25.114326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:25.114228    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"]
Oct 30 21:42:25.232681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:25.232642    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"]
Oct 30 21:42:25.749433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:25.749399    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 21:42:25.749649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:25.749453    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 21:42:26.345586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:26.345559    5786 scope.go:122] "RemoveContainer" containerID="48d20a1556bb1ec385bc32d7c03c64e330901738546fdb54714b19dfa9f51c66"
Oct 30 21:42:26.359115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:26.359094    5786 scope.go:122] "RemoveContainer" containerID="300d16c9db49257492e709e7687bb5df0136757bea43a59cfb415fccba13340a"
Oct 30 21:42:26.368905 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:26.368886    5786 scope.go:122] "RemoveContainer" containerID="14860fe0754bdba9bc9942a04906060a49a301b6e0898356000645085970d810"
Oct 30 21:42:26.378088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:26.378073    5786 scope.go:122] "RemoveContainer" containerID="bacd070c02c02b0d4e9caa1abdd9ee006c061e7ac7773e64caa8ca512c367866"
Oct 30 21:42:26.387304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:26.387286    5786 scope.go:122] "RemoveContainer" containerID="2ab3d25ed983556b10e22ad9af981704fa29c1f368f46d1a7655df97f01c5ea7"
Oct 30 21:42:26.782530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:26.782503    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:42:26.783820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:26.783800    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:42:30.115579 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:30.115499    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g" podUID="94afe0a5-f17a-4c0d-9cae-ee100ce85d11" containerName="payload-processing" containerID="cri-o://618e018327980aa9fbdb45fad6f24120dbc03f1feaa5574d142ecc66cec957a4" gracePeriod=25
Oct 30 21:42:30.234274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:30.234230    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt" podUID="7142d6be-6d9e-4802-8ce4-e0a2a29b187b" containerName="payload-pre-processing" containerID="cri-o://d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b" gracePeriod=25
Oct 30 21:42:34.033680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.033643    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main" containerID="cri-o://50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b" gracePeriod=45
Oct 30 21:42:34.034131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.034040    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main" containerID="cri-o://a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77" gracePeriod=45
Oct 30 21:42:34.342673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.342636    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:42:34.345616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.345595    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:42:34.414585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414564    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4066a185-16de-48f3-a35f-70f7220c7937-kube-api-access-mg2fn\" (UniqueName: \"kubernetes.io/projected/4066a185-16de-48f3-a35f-70f7220c7937-kube-api-access-mg2fn\") pod \"4066a185-16de-48f3-a35f-70f7220c7937\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") "
Oct 30 21:42:34.414703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414603    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-dshm\") pod \"95156f92-fddd-47a2-910b-838e7197a7d8\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") "
Oct 30 21:42:34.414703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414644    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95156f92-fddd-47a2-910b-838e7197a7d8-tls-certs\" (UniqueName: \"kubernetes.io/secret/95156f92-fddd-47a2-910b-838e7197a7d8-tls-certs\") pod \"95156f92-fddd-47a2-910b-838e7197a7d8\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") "
Oct 30 21:42:34.414703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414695    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-dshm\") pod \"4066a185-16de-48f3-a35f-70f7220c7937\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") "
Oct 30 21:42:34.414875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414728    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-model-cache\") pod \"4066a185-16de-48f3-a35f-70f7220c7937\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") "
Oct 30 21:42:34.414875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414751    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-home\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-home\") pod \"4066a185-16de-48f3-a35f-70f7220c7937\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") "
Oct 30 21:42:34.414875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414795    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-home\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-home\") pod \"95156f92-fddd-47a2-910b-838e7197a7d8\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") "
Oct 30 21:42:34.414875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414830    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4066a185-16de-48f3-a35f-70f7220c7937-tls-certs\" (UniqueName: \"kubernetes.io/secret/4066a185-16de-48f3-a35f-70f7220c7937-tls-certs\") pod \"4066a185-16de-48f3-a35f-70f7220c7937\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") "
Oct 30 21:42:34.415084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414880    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-model-cache\") pod \"95156f92-fddd-47a2-910b-838e7197a7d8\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") "
Oct 30 21:42:34.415084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414944    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-tmp-dir\") pod \"4066a185-16de-48f3-a35f-70f7220c7937\" (UID: \"4066a185-16de-48f3-a35f-70f7220c7937\") "
Oct 30 21:42:34.415084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.414964    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-tmp-dir\") pod \"95156f92-fddd-47a2-910b-838e7197a7d8\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") "
Oct 30 21:42:34.415277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.415124    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-model-cache" pod "4066a185-16de-48f3-a35f-70f7220c7937" (UID: "4066a185-16de-48f3-a35f-70f7220c7937"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:34.415277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.415183    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95156f92-fddd-47a2-910b-838e7197a7d8-kube-api-access-7gppg\" (UniqueName: \"kubernetes.io/projected/95156f92-fddd-47a2-910b-838e7197a7d8-kube-api-access-7gppg\") pod \"95156f92-fddd-47a2-910b-838e7197a7d8\" (UID: \"95156f92-fddd-47a2-910b-838e7197a7d8\") "
Oct 30 21:42:34.415649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.415624    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-home" pod "95156f92-fddd-47a2-910b-838e7197a7d8" (UID: "95156f92-fddd-47a2-910b-838e7197a7d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:34.416384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.416083    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.416384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.416111    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.416384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.416254    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-home" pod "4066a185-16de-48f3-a35f-70f7220c7937" (UID: "4066a185-16de-48f3-a35f-70f7220c7937"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:34.416384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.416270    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-tmp-dir" pod "4066a185-16de-48f3-a35f-70f7220c7937" (UID: "4066a185-16de-48f3-a35f-70f7220c7937"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:34.416384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.416322    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-model-cache" pod "95156f92-fddd-47a2-910b-838e7197a7d8" (UID: "95156f92-fddd-47a2-910b-838e7197a7d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:34.416722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.416425    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-tmp-dir" pod "95156f92-fddd-47a2-910b-838e7197a7d8" (UID: "95156f92-fddd-47a2-910b-838e7197a7d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:34.417300 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.417279    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-dshm" pod "95156f92-fddd-47a2-910b-838e7197a7d8" (UID: "95156f92-fddd-47a2-910b-838e7197a7d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:34.417393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.417343    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4066a185-16de-48f3-a35f-70f7220c7937-kube-api-access-mg2fn" pod "4066a185-16de-48f3-a35f-70f7220c7937" (UID: "4066a185-16de-48f3-a35f-70f7220c7937"). InnerVolumeSpecName "kube-api-access-mg2fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:34.417456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.417419    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95156f92-fddd-47a2-910b-838e7197a7d8-tls-certs" pod "95156f92-fddd-47a2-910b-838e7197a7d8" (UID: "95156f92-fddd-47a2-910b-838e7197a7d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:42:34.417598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.417586    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-dshm" pod "4066a185-16de-48f3-a35f-70f7220c7937" (UID: "4066a185-16de-48f3-a35f-70f7220c7937"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:34.417808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.417775    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95156f92-fddd-47a2-910b-838e7197a7d8-kube-api-access-7gppg" pod "95156f92-fddd-47a2-910b-838e7197a7d8" (UID: "95156f92-fddd-47a2-910b-838e7197a7d8"). InnerVolumeSpecName "kube-api-access-7gppg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:34.418154 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.418137    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4066a185-16de-48f3-a35f-70f7220c7937-tls-certs" pod "4066a185-16de-48f3-a35f-70f7220c7937" (UID: "4066a185-16de-48f3-a35f-70f7220c7937"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:42:34.517567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517535    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gppg\" (UniqueName: \"kubernetes.io/projected/95156f92-fddd-47a2-910b-838e7197a7d8-kube-api-access-7gppg\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517569    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg2fn\" (UniqueName: \"kubernetes.io/projected/4066a185-16de-48f3-a35f-70f7220c7937-kube-api-access-mg2fn\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517583    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517598    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95156f92-fddd-47a2-910b-838e7197a7d8-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517619    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517632    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517644    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4066a185-16de-48f3-a35f-70f7220c7937-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517657    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517696    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4066a185-16de-48f3-a35f-70f7220c7937-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:34.517754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:34.517711    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95156f92-fddd-47a2-910b-838e7197a7d8-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:35.098402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.098369    5786 generic.go:356] "Generic (PLEG): container finished" podID="4066a185-16de-48f3-a35f-70f7220c7937" containerID="50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b" exitCode=0
Oct 30 21:42:35.098966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.098408    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" event={"ID":"4066a185-16de-48f3-a35f-70f7220c7937","Type":"ContainerDied","Data":"50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b"}
Oct 30 21:42:35.099141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.098433    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"
Oct 30 21:42:35.099277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.098998    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj" event={"ID":"4066a185-16de-48f3-a35f-70f7220c7937","Type":"ContainerDied","Data":"b04945dfa7b882a00dcedea050e41f336ed7133d5dc6043c7fb8fadcf710a8da"}
Oct 30 21:42:35.099363 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.099318    5786 scope.go:122] "RemoveContainer" containerID="50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b"
Oct 30 21:42:35.101168 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.101148    5786 generic.go:356] "Generic (PLEG): container finished" podID="95156f92-fddd-47a2-910b-838e7197a7d8" containerID="a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77" exitCode=0
Oct 30 21:42:35.101274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.101198    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" event={"ID":"95156f92-fddd-47a2-910b-838e7197a7d8","Type":"ContainerDied","Data":"a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77"}
Oct 30 21:42:35.101274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.101249    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9" event={"ID":"95156f92-fddd-47a2-910b-838e7197a7d8","Type":"ContainerDied","Data":"10dc57cc0cb022da9a27abbe88e6b952d60f72e73029d9b24426da0f7076cc9c"}
Oct 30 21:42:35.101391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.101310    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"
Oct 30 21:42:35.112187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.112124    5786 scope.go:122] "RemoveContainer" containerID="50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b"
Oct 30 21:42:35.112469 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:42:35.112448    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b\": container with ID starting with 50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b not found: ID does not exist" containerID="50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b"
Oct 30 21:42:35.112601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.112483    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b"} err="failed to get container status \"50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b\": rpc error: code = NotFound desc = could not find container \"50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b\": container with ID starting with 50ee4ccee0e189f2912d7462f89ac0d2c4c85c9db0c86f2dc200be94c868de9b not found: ID does not exist"
Oct 30 21:42:35.112601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.112503    5786 scope.go:122] "RemoveContainer" containerID="a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77"
Oct 30 21:42:35.121420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.121399    5786 scope.go:122] "RemoveContainer" containerID="a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77"
Oct 30 21:42:35.121687 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:42:35.121669    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77\": container with ID starting with a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77 not found: ID does not exist" containerID="a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77"
Oct 30 21:42:35.121755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.121693    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77"} err="failed to get container status \"a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77\": rpc error: code = NotFound desc = could not find container \"a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77\": container with ID starting with a9c819504ebff463b79b5c4ae91a0142e6a17501f9789d5d1baf2f83749e3a77 not found: ID does not exist"
Oct 30 21:42:35.266726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.266699    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"]
Oct 30 21:42:35.315981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.315917    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-w4-847c62-kserve-995ff599-vcppj"]
Oct 30 21:42:35.385035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.385005    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"]
Oct 30 21:42:35.388506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:35.388482    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-847c62/distinct-2-w4-847c62-kserve-5cd44f4696-pwjz9"]
Oct 30 21:42:36.359976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:36.359946    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4066a185-16de-48f3-a35f-70f7220c7937" path="/var/lib/kubelet/pods/4066a185-16de-48f3-a35f-70f7220c7937/volumes"
Oct 30 21:42:36.360573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:36.360552    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" path="/var/lib/kubelet/pods/95156f92-fddd-47a2-910b-838e7197a7d8/volumes"
Oct 30 21:42:36.614288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:36.614190    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt" podUID="7142d6be-6d9e-4802-8ce4-e0a2a29b187b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:42:36.614451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:36.614278    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g" podUID="94afe0a5-f17a-4c0d-9cae-ee100ce85d11" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.968942    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"]
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970056    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7128c5ce-3dcc-4743-ad9b-f63b53a2b523" containerName="revoke-keys"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970079    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7128c5ce-3dcc-4743-ad9b-f63b53a2b523" containerName="revoke-keys"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970120    5786 cpu_manager.go:414] "containerMap: removing container" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970127    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970140    5786 cpu_manager.go:414] "containerMap: removing container" podUID="aead0097-88ae-4970-81c2-f856c8000169" containerName="payload-processing"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970147    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aead0097-88ae-4970-81c2-f856c8000169" containerName="payload-processing"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970172    5786 cpu_manager.go:414] "containerMap: removing container" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970180    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970251    5786 cpu_manager.go:414] "containerMap: removing container" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970259    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f382dea-dd70-4ebc-accb-a9bab6fc577f" containerName="istio-proxy"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970270    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e244748d-aecf-4ab3-973f-d0eae1307c1c" containerName="payload-pre-processing"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970276    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e244748d-aecf-4ab3-973f-d0eae1307c1c" containerName="payload-pre-processing"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970477    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7128c5ce-3dcc-4743-ad9b-f63b53a2b523" containerName="revoke-keys"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970497    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="95156f92-fddd-47a2-910b-838e7197a7d8" containerName="main"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970506    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="aead0097-88ae-4970-81c2-f856c8000169" containerName="payload-processing"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970520    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="4066a185-16de-48f3-a35f-70f7220c7937" containerName="main"
Oct 30 21:42:37.973245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:37.970539    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e244748d-aecf-4ab3-973f-d0eae1307c1c" containerName="payload-pre-processing"
Oct 30 21:42:38.028353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.028320    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.116865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.116816    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"]
Oct 30 21:42:38.121562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.121534    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-5f31bd-openshift-default-dockercfg-hddb8\""
Oct 30 21:42:38.220136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220109    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-envoy\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.220328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220160    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istiod-ca-cert\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.220328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220255    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-data\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.220328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220289    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-token\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.220474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220387    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-certs\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.220474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220439    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-podinfo\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.220554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220513    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-socket\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.220611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220574    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqlh9\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-kube-api-access-bqlh9\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.220664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.220651    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-credential-socket\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.321932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.321870    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-socket\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.321932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.321905    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqlh9\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-kube-api-access-bqlh9\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.321937    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-credential-socket\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322003    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-envoy\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322043    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istiod-ca-cert\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322068    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-data\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322084    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-token\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322155    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-certs\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322236    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-podinfo\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322319    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-socket\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322350    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-credential-socket\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322467    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-data\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322590    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-certs\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.322794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.322772    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istiod-ca-cert\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.324465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.324443    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-envoy\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.324775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.324758    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-podinfo\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.380862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.380839    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-token\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.381054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.381033    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqlh9\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-kube-api-access-bqlh9\") pod \"e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") " pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.442026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.441969    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:38.911263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.911234    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"]
Oct 30 21:42:38.941552 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:42:38.941523    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ddcb89a_a9a5_4798_b1c0_24d7da39df87.slice/crio-0a60d3118b1a0858ffb9d2b712021f0afc13487adec0f7dd5332ac32c83b4565 WatchSource:0}: Error finding container 0a60d3118b1a0858ffb9d2b712021f0afc13487adec0f7dd5332ac32c83b4565: Status 404 returned error can't find the container with id 0a60d3118b1a0858ffb9d2b712021f0afc13487adec0f7dd5332ac32c83b4565
Oct 30 21:42:38.944328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.944291    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:42:38.944421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.944395    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:42:38.944456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:38.944437    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:42:39.151430 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:39.151394    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" event={"ID":"0ddcb89a-a9a5-4798-b1c0-24d7da39df87","Type":"ContainerStarted","Data":"53373e78d3d418c17cbf22bade2d68fa8be6a6610ec5ae3488f1bb2b31175fbc"}
Oct 30 21:42:39.151875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:39.151447    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" event={"ID":"0ddcb89a-a9a5-4798-b1c0-24d7da39df87","Type":"ContainerStarted","Data":"0a60d3118b1a0858ffb9d2b712021f0afc13487adec0f7dd5332ac32c83b4565"}
Oct 30 21:42:39.284645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:39.284600    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" podStartSLOduration=2.2845827930000002 podStartE2EDuration="2.284582793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:39.284229629 +0000 UTC m=+1815.155348772" watchObservedRunningTime="2026-10-30 21:42:39.284582793 +0000 UTC m=+1815.155701913"
Oct 30 21:42:39.447235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:39.446459    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:40.449004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:40.448955    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:41.165775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:41.165731    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:41.166548 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:41.166528    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:42:46.615062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:46.615025    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt" podUID="7142d6be-6d9e-4802-8ce4-e0a2a29b187b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:42:46.615062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:46.615056    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g" podUID="94afe0a5-f17a-4c0d-9cae-ee100ce85d11" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:42:47.915277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:47.915246    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"]
Oct 30 21:42:47.919745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:47.919725    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.034246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.034194    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.034401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.034365    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w274h\" (UniqueName: \"kubernetes.io/projected/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-kube-api-access-w274h\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.034564 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.034542    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.035086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.035063    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:42:48.035765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.035746    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:42:48.040872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.040846    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"]
Oct 30 21:42:48.046271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.046242    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"]
Oct 30 21:42:48.046385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.046329    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.076567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.076547    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"]
Oct 30 21:42:48.166542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.163104    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wqlw\" (UniqueName: \"kubernetes.io/projected/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-kube-api-access-2wqlw\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.168398 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.168361    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.169268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.169235    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.171668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.171650    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.171871 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.171853    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.172010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.171993    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w274h\" (UniqueName: \"kubernetes.io/projected/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-kube-api-access-w274h\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.172735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.171471    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.176389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.176370    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.233722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.233690    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w274h\" (UniqueName: \"kubernetes.io/projected/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-kube-api-access-w274h\") pod \"maas-api-revoke-sub-9b4693dc-b649w\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.233940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.233914    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:48.273099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.273073    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wqlw\" (UniqueName: \"kubernetes.io/projected/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-kube-api-access-2wqlw\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.273283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.273228    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.273283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.273267    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.273813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.273789    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.276247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.276083    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.335858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.335817    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wqlw\" (UniqueName: \"kubernetes.io/projected/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-kube-api-access-2wqlw\") pod \"maas-api-revoke-sub-04c9cda6-b8c8w\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.359723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.359620    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:48.778278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.777450    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"]
Oct 30 21:42:48.784838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:48.782565    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"]
Oct 30 21:42:49.020091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.020062    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"]
Oct 30 21:42:49.025735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.025716    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:49.089761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.089741    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-5f31bd\""
Oct 30 21:42:49.089873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.089772    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-5f31bd-dockercfg-jw6cv\""
Oct 30 21:42:49.092969 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.092944    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzgpv\" (UniqueName: \"kubernetes.io/projected/1a2feebb-1715-45b3-b8e5-6c808b94c262-kube-api-access-xzgpv\") pod \"payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp\" (UID: \"1a2feebb-1715-45b3-b8e5-6c808b94c262\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:49.093157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.093132    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a2feebb-1715-45b3-b8e5-6c808b94c262-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp\" (UID: \"1a2feebb-1715-45b3-b8e5-6c808b94c262\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:49.096377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.096359    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"]
Oct 30 21:42:49.328305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.327912    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzgpv\" (UniqueName: \"kubernetes.io/projected/1a2feebb-1715-45b3-b8e5-6c808b94c262-kube-api-access-xzgpv\") pod \"payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp\" (UID: \"1a2feebb-1715-45b3-b8e5-6c808b94c262\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:49.328305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.328056    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a2feebb-1715-45b3-b8e5-6c808b94c262-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp\" (UID: \"1a2feebb-1715-45b3-b8e5-6c808b94c262\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:49.328973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.328947    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a2feebb-1715-45b3-b8e5-6c808b94c262-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp\" (UID: \"1a2feebb-1715-45b3-b8e5-6c808b94c262\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:49.346055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.346032    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"]
Oct 30 21:42:49.353779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.352700    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"]
Oct 30 21:42:49.353779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.352772    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w" event={"ID":"a96bdf00-65f8-4081-9b0e-7d4b6345b7be","Type":"ContainerStarted","Data":"90a80049634c79679ecf73c26176f03a8d163d25be1cef46dd8da1c3097f2d83"}
Oct 30 21:42:49.353779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.352797    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w" event={"ID":"117d7cfc-33d9-4d6c-b84f-dcedead30bc9","Type":"ContainerStarted","Data":"608a13bc9d104fd3dfbcb5bf08e8ac353ecf40d28cc602fd94221f48bbdef696"}
Oct 30 21:42:49.353779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.352821    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:49.389132 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.389113    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzgpv\" (UniqueName: \"kubernetes.io/projected/1a2feebb-1715-45b3-b8e5-6c808b94c262-kube-api-access-xzgpv\") pod \"payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp\" (UID: \"1a2feebb-1715-45b3-b8e5-6c808b94c262\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:49.429483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.429457    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d006ce-9d62-4a69-bed0-3e2d89522c0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg\" (UID: \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:49.429653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.429637    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65jrs\" (UniqueName: \"kubernetes.io/projected/49d006ce-9d62-4a69-bed0-3e2d89522c0b-kube-api-access-65jrs\") pod \"payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg\" (UID: \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:49.531427 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.531395    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d006ce-9d62-4a69-bed0-3e2d89522c0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg\" (UID: \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:49.531593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.531518    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65jrs\" (UniqueName: \"kubernetes.io/projected/49d006ce-9d62-4a69-bed0-3e2d89522c0b-kube-api-access-65jrs\") pod \"payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg\" (UID: \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:49.532055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.532032    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d006ce-9d62-4a69-bed0-3e2d89522c0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg\" (UID: \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:49.591264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.591168    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65jrs\" (UniqueName: \"kubernetes.io/projected/49d006ce-9d62-4a69-bed0-3e2d89522c0b-kube-api-access-65jrs\") pod \"payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg\" (UID: \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:49.638625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.638602    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:49.673735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:49.673708    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:50.089518 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:42:50.089488    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2feebb_1715_45b3_b8e5_6c808b94c262.slice/crio-96a57452706245c6db140c99303470546a60f16b1d80856b764e4555abaae202 WatchSource:0}: Error finding container 96a57452706245c6db140c99303470546a60f16b1d80856b764e4555abaae202: Status 404 returned error can't find the container with id 96a57452706245c6db140c99303470546a60f16b1d80856b764e4555abaae202
Oct 30 21:42:50.089838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.089542    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"]
Oct 30 21:42:50.104557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.104536    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"]
Oct 30 21:42:50.131094 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:42:50.131071    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d006ce_9d62_4a69_bed0_3e2d89522c0b.slice/crio-f22c6432fcd13bf25ecd420d992c7c2e0fea346aee8e5e7cc5a0e753a2a17f9e WatchSource:0}: Error finding container f22c6432fcd13bf25ecd420d992c7c2e0fea346aee8e5e7cc5a0e753a2a17f9e: Status 404 returned error can't find the container with id f22c6432fcd13bf25ecd420d992c7c2e0fea346aee8e5e7cc5a0e753a2a17f9e
Oct 30 21:42:50.365361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.364902    5786 generic.go:356] "Generic (PLEG): container finished" podID="a96bdf00-65f8-4081-9b0e-7d4b6345b7be" containerID="56a4115114e811792d766a0fd6cd95dff6b0f2976f8a1aa657c6a8877be58ac0" exitCode=0
Oct 30 21:42:50.368614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.368579    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w" event={"ID":"a96bdf00-65f8-4081-9b0e-7d4b6345b7be","Type":"ContainerDied","Data":"56a4115114e811792d766a0fd6cd95dff6b0f2976f8a1aa657c6a8877be58ac0"}
Oct 30 21:42:50.368733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.368651    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:42:50.368733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.368724    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp" event={"ID":"1a2feebb-1715-45b3-b8e5-6c808b94c262","Type":"ContainerStarted","Data":"5dd672e4281b230e16677f37dde8e025568eb3fca0640c4385821e8402cc88fd"}
Oct 30 21:42:50.368849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.368746    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp" event={"ID":"1a2feebb-1715-45b3-b8e5-6c808b94c262","Type":"ContainerStarted","Data":"96a57452706245c6db140c99303470546a60f16b1d80856b764e4555abaae202"}
Oct 30 21:42:50.370110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.370075    5786 generic.go:356] "Generic (PLEG): container finished" podID="117d7cfc-33d9-4d6c-b84f-dcedead30bc9" containerID="be8b7ec82be7e3ddb4dab3bff8a8ea565bb6321ca7d7c24effcf7a6605b2925b" exitCode=0
Oct 30 21:42:50.370364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.370258    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w" event={"ID":"117d7cfc-33d9-4d6c-b84f-dcedead30bc9","Type":"ContainerDied","Data":"be8b7ec82be7e3ddb4dab3bff8a8ea565bb6321ca7d7c24effcf7a6605b2925b"}
Oct 30 21:42:50.372140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.372121    5786 generic.go:356] "Generic (PLEG): container finished" podID="94afe0a5-f17a-4c0d-9cae-ee100ce85d11" containerID="618e018327980aa9fbdb45fad6f24120dbc03f1feaa5574d142ecc66cec957a4" exitCode=0
Oct 30 21:42:50.372264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.372191    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g" event={"ID":"94afe0a5-f17a-4c0d-9cae-ee100ce85d11","Type":"ContainerDied","Data":"618e018327980aa9fbdb45fad6f24120dbc03f1feaa5574d142ecc66cec957a4"}
Oct 30 21:42:50.374187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.374166    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg" event={"ID":"49d006ce-9d62-4a69-bed0-3e2d89522c0b","Type":"ContainerStarted","Data":"85883410d664f62981d4a851a37d99339a1eb93e55eec37291fe836dd6a0fc3c"}
Oct 30 21:42:50.374307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.374228    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg" event={"ID":"49d006ce-9d62-4a69-bed0-3e2d89522c0b","Type":"ContainerStarted","Data":"f22c6432fcd13bf25ecd420d992c7c2e0fea346aee8e5e7cc5a0e753a2a17f9e"}
Oct 30 21:42:50.374351 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.374316    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:42:50.422874 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.422847    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:42:50.445821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.445787    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-kube-api-access-5hb2g\" (UniqueName: \"kubernetes.io/projected/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-kube-api-access-5hb2g\") pod \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\" (UID: \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\") "
Oct 30 21:42:50.446321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.446295    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-plugins-config-volume\") pod \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\" (UID: \"94afe0a5-f17a-4c0d-9cae-ee100ce85d11\") "
Oct 30 21:42:50.447901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.447872    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-plugins-config-volume" pod "94afe0a5-f17a-4c0d-9cae-ee100ce85d11" (UID: "94afe0a5-f17a-4c0d-9cae-ee100ce85d11"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:50.452589 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.452553    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-kube-api-access-5hb2g" pod "94afe0a5-f17a-4c0d-9cae-ee100ce85d11" (UID: "94afe0a5-f17a-4c0d-9cae-ee100ce85d11"). InnerVolumeSpecName "kube-api-access-5hb2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:50.509194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.509168    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:42:50.548464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.548441    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-plugins-config-volume\") pod \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\" (UID: \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\") "
Oct 30 21:42:50.548621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.548609    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-kube-api-access-27bmt\" (UniqueName: \"kubernetes.io/projected/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-kube-api-access-27bmt\") pod \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\" (UID: \"7142d6be-6d9e-4802-8ce4-e0a2a29b187b\") "
Oct 30 21:42:50.548760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.548740    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-plugins-config-volume" pod "7142d6be-6d9e-4802-8ce4-e0a2a29b187b" (UID: "7142d6be-6d9e-4802-8ce4-e0a2a29b187b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:50.548991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.548977    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hb2g\" (UniqueName: \"kubernetes.io/projected/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-kube-api-access-5hb2g\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:50.549033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.548995    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94afe0a5-f17a-4c0d-9cae-ee100ce85d11-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:50.549033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.549004    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:50.550661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.550643    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-kube-api-access-27bmt" pod "7142d6be-6d9e-4802-8ce4-e0a2a29b187b" (UID: "7142d6be-6d9e-4802-8ce4-e0a2a29b187b"). InnerVolumeSpecName "kube-api-access-27bmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:50.650530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.650508    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27bmt\" (UniqueName: \"kubernetes.io/projected/7142d6be-6d9e-4802-8ce4-e0a2a29b187b-kube-api-access-27bmt\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:50.656669 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:50.656623    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg" podStartSLOduration=1.65661196 podStartE2EDuration="1.65661196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:50.651798257 +0000 UTC m=+1826.522917375" watchObservedRunningTime="2026-10-30 21:42:50.65661196 +0000 UTC m=+1826.527731120"
Oct 30 21:42:51.045695 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.045649    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp" podStartSLOduration=3.045634894 podStartE2EDuration="3.045634894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:51.045303048 +0000 UTC m=+1826.916422167" watchObservedRunningTime="2026-10-30 21:42:51.045634894 +0000 UTC m=+1826.916754037"
Oct 30 21:42:51.386013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.385943    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"
Oct 30 21:42:51.386013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.385975    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g" event={"ID":"94afe0a5-f17a-4c0d-9cae-ee100ce85d11","Type":"ContainerDied","Data":"13e50d59ffccf842025f3dcbfc7f86fcc8922ecb15616f7b94819562cfbdae2c"}
Oct 30 21:42:51.386498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.386032    5786 scope.go:122] "RemoveContainer" containerID="618e018327980aa9fbdb45fad6f24120dbc03f1feaa5574d142ecc66cec957a4"
Oct 30 21:42:51.387512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.387492    5786 generic.go:356] "Generic (PLEG): container finished" podID="7142d6be-6d9e-4802-8ce4-e0a2a29b187b" containerID="d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b" exitCode=0
Oct 30 21:42:51.387639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.387542    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"
Oct 30 21:42:51.387639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.387588    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt" event={"ID":"7142d6be-6d9e-4802-8ce4-e0a2a29b187b","Type":"ContainerDied","Data":"d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b"}
Oct 30 21:42:51.387639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.387631    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt" event={"ID":"7142d6be-6d9e-4802-8ce4-e0a2a29b187b","Type":"ContainerDied","Data":"e8867f8e27f1a0d83d04c2b973914acfe2c21f7915fc6d6994a7de36cd1286ad"}
Oct 30 21:42:51.398815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.398799    5786 scope.go:122] "RemoveContainer" containerID="d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b"
Oct 30 21:42:51.408805 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.408767    5786 scope.go:122] "RemoveContainer" containerID="d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b"
Oct 30 21:42:51.409144 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:42:51.409115    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b\": container with ID starting with d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b not found: ID does not exist" containerID="d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b"
Oct 30 21:42:51.409291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.409150    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b"} err="failed to get container status \"d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b\": rpc error: code = NotFound desc = could not find container \"d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b\": container with ID starting with d82c6be80e6033c963a4c1edcfc62fde695747a9e177c231b878686e1291880b not found: ID does not exist"
Oct 30 21:42:51.570490 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.570463    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"]
Oct 30 21:42:51.575514 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.575485    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-847c62-78849c7ff-5p59g"]
Oct 30 21:42:51.584188 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.584168    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:51.586313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.586298    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:51.665492 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.665472    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-kube-api-access-w274h\" (UniqueName: \"kubernetes.io/projected/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-kube-api-access-w274h\") pod \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") "
Oct 30 21:42:51.665599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.665590    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-openshift-ca\") pod \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") "
Oct 30 21:42:51.665677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.665661    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-xks-ca\") pod \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") "
Oct 30 21:42:51.665792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.665775    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-kube-api-access-2wqlw\" (UniqueName: \"kubernetes.io/projected/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-kube-api-access-2wqlw\") pod \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\" (UID: \"117d7cfc-33d9-4d6c-b84f-dcedead30bc9\") "
Oct 30 21:42:51.665879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.665864    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-openshift-ca\") pod \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") "
Oct 30 21:42:51.665934 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.665912    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-openshift-ca" pod "117d7cfc-33d9-4d6c-b84f-dcedead30bc9" (UID: "117d7cfc-33d9-4d6c-b84f-dcedead30bc9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:51.665986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.665916    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-xks-ca\") pod \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\" (UID: \"a96bdf00-65f8-4081-9b0e-7d4b6345b7be\") "
Oct 30 21:42:51.666255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.666230    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-openshift-ca" pod "a96bdf00-65f8-4081-9b0e-7d4b6345b7be" (UID: "a96bdf00-65f8-4081-9b0e-7d4b6345b7be"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:51.666917 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.666893    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:51.667025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.666924    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:51.667859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.667834    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-xks-ca" pod "117d7cfc-33d9-4d6c-b84f-dcedead30bc9" (UID: "117d7cfc-33d9-4d6c-b84f-dcedead30bc9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:42:51.667932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.667867    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-kube-api-access-w274h" pod "a96bdf00-65f8-4081-9b0e-7d4b6345b7be" (UID: "a96bdf00-65f8-4081-9b0e-7d4b6345b7be"). InnerVolumeSpecName "kube-api-access-w274h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:51.667979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.667950    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-kube-api-access-2wqlw" pod "117d7cfc-33d9-4d6c-b84f-dcedead30bc9" (UID: "117d7cfc-33d9-4d6c-b84f-dcedead30bc9"). InnerVolumeSpecName "kube-api-access-2wqlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:51.668131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.668112    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-xks-ca" pod "a96bdf00-65f8-4081-9b0e-7d4b6345b7be" (UID: "a96bdf00-65f8-4081-9b0e-7d4b6345b7be"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:42:51.715461 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.715440    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"]
Oct 30 21:42:51.731021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.730999    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-847c62-77dd9474c9-r6rpt"]
Oct 30 21:42:51.768764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.768743    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:51.768849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.768766    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wqlw\" (UniqueName: \"kubernetes.io/projected/117d7cfc-33d9-4d6c-b84f-dcedead30bc9-kube-api-access-2wqlw\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:51.768849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.768778    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:51.768849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:51.768787    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w274h\" (UniqueName: \"kubernetes.io/projected/a96bdf00-65f8-4081-9b0e-7d4b6345b7be-kube-api-access-w274h\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:42:52.355924 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:42:52.355885    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:42:52.362600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:52.362571    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7142d6be-6d9e-4802-8ce4-e0a2a29b187b" path="/var/lib/kubelet/pods/7142d6be-6d9e-4802-8ce4-e0a2a29b187b/volumes"
Oct 30 21:42:52.363081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:52.363061    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94afe0a5-f17a-4c0d-9cae-ee100ce85d11" path="/var/lib/kubelet/pods/94afe0a5-f17a-4c0d-9cae-ee100ce85d11/volumes"
Oct 30 21:42:52.396125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:52.396093    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"
Oct 30 21:42:52.396464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:52.396120    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w" event={"ID":"117d7cfc-33d9-4d6c-b84f-dcedead30bc9","Type":"ContainerDied","Data":"608a13bc9d104fd3dfbcb5bf08e8ac353ecf40d28cc602fd94221f48bbdef696"}
Oct 30 21:42:52.396464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:52.396185    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="608a13bc9d104fd3dfbcb5bf08e8ac353ecf40d28cc602fd94221f48bbdef696"
Oct 30 21:42:52.399785 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:52.399762    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w" event={"ID":"a96bdf00-65f8-4081-9b0e-7d4b6345b7be","Type":"ContainerDied","Data":"90a80049634c79679ecf73c26176f03a8d163d25be1cef46dd8da1c3097f2d83"}
Oct 30 21:42:52.399908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:52.399790    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"
Oct 30 21:42:52.399908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:52.399798    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90a80049634c79679ecf73c26176f03a8d163d25be1cef46dd8da1c3097f2d83"
Oct 30 21:42:55.732231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.728825    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"]
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.737700    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7142d6be-6d9e-4802-8ce4-e0a2a29b187b" containerName="payload-pre-processing"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.737728    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7142d6be-6d9e-4802-8ce4-e0a2a29b187b" containerName="payload-pre-processing"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.737744    5786 cpu_manager.go:414] "containerMap: removing container" podUID="94afe0a5-f17a-4c0d-9cae-ee100ce85d11" containerName="payload-processing"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.737752    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94afe0a5-f17a-4c0d-9cae-ee100ce85d11" containerName="payload-processing"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.737789    5786 cpu_manager.go:414] "containerMap: removing container" podUID="117d7cfc-33d9-4d6c-b84f-dcedead30bc9" containerName="revoke-keys"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.737796    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="117d7cfc-33d9-4d6c-b84f-dcedead30bc9" containerName="revoke-keys"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.737841    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a96bdf00-65f8-4081-9b0e-7d4b6345b7be" containerName="revoke-keys"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.737847    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a96bdf00-65f8-4081-9b0e-7d4b6345b7be" containerName="revoke-keys"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.738039    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="117d7cfc-33d9-4d6c-b84f-dcedead30bc9" containerName="revoke-keys"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.738069    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a96bdf00-65f8-4081-9b0e-7d4b6345b7be" containerName="revoke-keys"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.738086    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7142d6be-6d9e-4802-8ce4-e0a2a29b187b" containerName="payload-pre-processing"
Oct 30 21:42:55.744315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.738097    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="94afe0a5-f17a-4c0d-9cae-ee100ce85d11" containerName="payload-processing"
Oct 30 21:42:55.759229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.754746    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:55.839339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.839307    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25g5b\" (UniqueName: \"kubernetes.io/projected/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-kube-api-access-25g5b\") pod \"payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z\" (UID: \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:55.839545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.839521    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z\" (UID: \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:55.877893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.877870    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-e0f967\""
Oct 30 21:42:55.878024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.877959    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-e0f967-dockercfg-4trww\""
Oct 30 21:42:55.881134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.881091    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"]
Oct 30 21:42:55.910545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.910523    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"]
Oct 30 21:42:55.915055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.915038    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:55.940405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.940385    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z\" (UID: \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:55.940533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.940513    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a35a96-a63c-4294-88e8-0203a394417c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp\" (UID: \"a4a35a96-a63c-4294-88e8-0203a394417c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:55.940605 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.940561    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlmvk\" (UniqueName: \"kubernetes.io/projected/a4a35a96-a63c-4294-88e8-0203a394417c-kube-api-access-qlmvk\") pod \"payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp\" (UID: \"a4a35a96-a63c-4294-88e8-0203a394417c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:55.940670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.940634    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25g5b\" (UniqueName: \"kubernetes.io/projected/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-kube-api-access-25g5b\") pod \"payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z\" (UID: \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:55.941126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:55.941098    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z\" (UID: \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:56.005386 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.005334    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25g5b\" (UniqueName: \"kubernetes.io/projected/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-kube-api-access-25g5b\") pod \"payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z\" (UID: \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:56.008436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.008415    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"]
Oct 30 21:42:56.043341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.042707    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a35a96-a63c-4294-88e8-0203a394417c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp\" (UID: \"a4a35a96-a63c-4294-88e8-0203a394417c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:56.043471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.043426    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlmvk\" (UniqueName: \"kubernetes.io/projected/a4a35a96-a63c-4294-88e8-0203a394417c-kube-api-access-qlmvk\") pod \"payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp\" (UID: \"a4a35a96-a63c-4294-88e8-0203a394417c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:56.044084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.044053    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a35a96-a63c-4294-88e8-0203a394417c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp\" (UID: \"a4a35a96-a63c-4294-88e8-0203a394417c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:56.082354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.082332    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:56.105677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.105655    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlmvk\" (UniqueName: \"kubernetes.io/projected/a4a35a96-a63c-4294-88e8-0203a394417c-kube-api-access-qlmvk\") pod \"payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp\" (UID: \"a4a35a96-a63c-4294-88e8-0203a394417c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:56.229435 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.229405    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:56.485930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.485903    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"]
Oct 30 21:42:56.487079 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:42:56.487040    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308f38d8_cf2d_4dff_ab54_a2c36e36f3e8.slice/crio-aa227eeb9036d5f690464a5f6283640021a4c85f61d25b79949ec59b41f05c95 WatchSource:0}: Error finding container aa227eeb9036d5f690464a5f6283640021a4c85f61d25b79949ec59b41f05c95: Status 404 returned error can't find the container with id aa227eeb9036d5f690464a5f6283640021a4c85f61d25b79949ec59b41f05c95
Oct 30 21:42:56.621972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:56.621949    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"]
Oct 30 21:42:56.622077 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:42:56.621986    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a35a96_a63c_4294_88e8_0203a394417c.slice/crio-a568cd38ea107093a195a2629ea5d495fb8e531ded59830b351d7b6b2d69ee61 WatchSource:0}: Error finding container a568cd38ea107093a195a2629ea5d495fb8e531ded59830b351d7b6b2d69ee61: Status 404 returned error can't find the container with id a568cd38ea107093a195a2629ea5d495fb8e531ded59830b351d7b6b2d69ee61
Oct 30 21:42:57.435853 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:57.435820    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z" event={"ID":"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8","Type":"ContainerStarted","Data":"1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123"}
Oct 30 21:42:57.436357 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:57.435868    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z" event={"ID":"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8","Type":"ContainerStarted","Data":"aa227eeb9036d5f690464a5f6283640021a4c85f61d25b79949ec59b41f05c95"}
Oct 30 21:42:57.436357 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:57.435908    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:42:57.437501 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:57.437477    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp" event={"ID":"a4a35a96-a63c-4294-88e8-0203a394417c","Type":"ContainerStarted","Data":"a434860fc45602662630c8b6e85451c2a599c79312ce0c02d950d81f6508b5f2"}
Oct 30 21:42:57.437607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:57.437520    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp" event={"ID":"a4a35a96-a63c-4294-88e8-0203a394417c","Type":"ContainerStarted","Data":"a568cd38ea107093a195a2629ea5d495fb8e531ded59830b351d7b6b2d69ee61"}
Oct 30 21:42:57.437607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:57.437558    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:42:57.577937 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:42:57.577897    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z" podStartSLOduration=2.577884338 podStartE2EDuration="2.577884338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:57.575163807 +0000 UTC m=+1833.446282937" watchObservedRunningTime="2026-10-30 21:42:57.577884338 +0000 UTC m=+1833.449003451"
Oct 30 21:43:01.144092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.144043    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp" podStartSLOduration=6.144028951 podStartE2EDuration="6.144028951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:57.719654691 +0000 UTC m=+1833.590773812" watchObservedRunningTime="2026-10-30 21:43:01.144028951 +0000 UTC m=+1837.015148070"
Oct 30 21:43:01.145749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.145728    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"]
Oct 30 21:43:01.150281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.150263    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:01.218702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.213131    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-99003c\""
Oct 30 21:43:01.227230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.224178    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsjzk\" (UniqueName: \"kubernetes.io/projected/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-kube-api-access-wsjzk\") pod \"payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j\" (UID: \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:01.227230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.224689    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j\" (UID: \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:01.276230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.274421    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-99003c-dockercfg-fs7fb\""
Oct 30 21:43:01.290022 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.285033    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"]
Oct 30 21:43:01.337230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.335027    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j\" (UID: \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:01.337230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.335265    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsjzk\" (UniqueName: \"kubernetes.io/projected/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-kube-api-access-wsjzk\") pod \"payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j\" (UID: \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:01.343277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.343141    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j\" (UID: \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:01.384239 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.383984    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"]
Oct 30 21:43:01.400800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.400721    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:43:01.400979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.400804    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:01.402365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.402339    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:43:01.404754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.404726    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsjzk\" (UniqueName: \"kubernetes.io/projected/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-kube-api-access-wsjzk\") pod \"payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j\" (UID: \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:01.442456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.442429    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"]
Oct 30 21:43:01.464883 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.464845    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:01.538728 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.538696    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc71b1a2-432d-48ad-a8a9-760c530e032d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c\" (UID: \"bc71b1a2-432d-48ad-a8a9-760c530e032d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:01.538870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.538818    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qndl\" (UniqueName: \"kubernetes.io/projected/bc71b1a2-432d-48ad-a8a9-760c530e032d-kube-api-access-9qndl\") pod \"payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c\" (UID: \"bc71b1a2-432d-48ad-a8a9-760c530e032d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:01.648632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.648590    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc71b1a2-432d-48ad-a8a9-760c530e032d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c\" (UID: \"bc71b1a2-432d-48ad-a8a9-760c530e032d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:01.648800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.648708    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qndl\" (UniqueName: \"kubernetes.io/projected/bc71b1a2-432d-48ad-a8a9-760c530e032d-kube-api-access-9qndl\") pod \"payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c\" (UID: \"bc71b1a2-432d-48ad-a8a9-760c530e032d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:01.649312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.649290    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc71b1a2-432d-48ad-a8a9-760c530e032d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c\" (UID: \"bc71b1a2-432d-48ad-a8a9-760c530e032d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:01.710646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.710617    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qndl\" (UniqueName: \"kubernetes.io/projected/bc71b1a2-432d-48ad-a8a9-760c530e032d-kube-api-access-9qndl\") pod \"payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c\" (UID: \"bc71b1a2-432d-48ad-a8a9-760c530e032d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:01.728946 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.728924    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:01.868008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:01.867981    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"]
Oct 30 21:43:01.868301 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:01.868273    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6bfa334_afee_4d0d_b43b_bf860dae5f1d.slice/crio-cdf180c7abf7572ae4a30da193746fc932777e5d44487073496ab041feebb7ee WatchSource:0}: Error finding container cdf180c7abf7572ae4a30da193746fc932777e5d44487073496ab041feebb7ee: Status 404 returned error can't find the container with id cdf180c7abf7572ae4a30da193746fc932777e5d44487073496ab041feebb7ee
Oct 30 21:43:02.123053 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:02.123030    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"]
Oct 30 21:43:02.125161 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:02.125134    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc71b1a2_432d_48ad_a8a9_760c530e032d.slice/crio-edb07943860fab5c9d20e6cc1917b250fccb7bd7edfdaf7982da63b3fd2e1828 WatchSource:0}: Error finding container edb07943860fab5c9d20e6cc1917b250fccb7bd7edfdaf7982da63b3fd2e1828: Status 404 returned error can't find the container with id edb07943860fab5c9d20e6cc1917b250fccb7bd7edfdaf7982da63b3fd2e1828
Oct 30 21:43:02.480066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:02.480029    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j" event={"ID":"d6bfa334-afee-4d0d-b43b-bf860dae5f1d","Type":"ContainerStarted","Data":"cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf"}
Oct 30 21:43:02.480544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:02.480091    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j" event={"ID":"d6bfa334-afee-4d0d-b43b-bf860dae5f1d","Type":"ContainerStarted","Data":"cdf180c7abf7572ae4a30da193746fc932777e5d44487073496ab041feebb7ee"}
Oct 30 21:43:02.480544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:02.480229    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:02.481846 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:02.481783    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c" event={"ID":"bc71b1a2-432d-48ad-a8a9-760c530e032d","Type":"ContainerStarted","Data":"62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456"}
Oct 30 21:43:02.481846 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:02.481819    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c" event={"ID":"bc71b1a2-432d-48ad-a8a9-760c530e032d","Type":"ContainerStarted","Data":"edb07943860fab5c9d20e6cc1917b250fccb7bd7edfdaf7982da63b3fd2e1828"}
Oct 30 21:43:02.481846 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:02.481842    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:02.629145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:02.629106    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j" podStartSLOduration=1.629089966 podStartE2EDuration="1.629089966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:02.627797294 +0000 UTC m=+1838.498916415" watchObservedRunningTime="2026-10-30 21:43:02.629089966 +0000 UTC m=+1838.500209085"
Oct 30 21:43:08.447178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:08.447150    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:43:08.447658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:08.447239    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:43:08.590746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:08.590698    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c" podStartSLOduration=7.590685483 podStartE2EDuration="7.590685483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:02.76970387 +0000 UTC m=+1838.640822993" watchObservedRunningTime="2026-10-30 21:43:08.590685483 +0000 UTC m=+1844.461804595"
Oct 30 21:43:13.234199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.234158    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"]
Oct 30 21:43:13.238863 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.238847    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.292672 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.292643    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:43:13.317695 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.317662    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.317818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.317789    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.317898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.317881    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7v7f\" (UniqueName: \"kubernetes.io/projected/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-kube-api-access-p7v7f\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.341443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.341419    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:43:13.345280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.345256    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"]
Oct 30 21:43:13.419060 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.419035    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.419166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.419111    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.419264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.419177    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7v7f\" (UniqueName: \"kubernetes.io/projected/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-kube-api-access-p7v7f\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.419761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.419741    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.421649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.421631    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.478536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.478512    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7v7f\" (UniqueName: \"kubernetes.io/projected/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-kube-api-access-p7v7f\") pod \"maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.492879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.492823    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:43:13.492963 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.492900    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:43:13.553595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.553577    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:13.934269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:13.934239    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"]
Oct 30 21:43:13.935393 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:13.935362    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79cbc4ba_4b7c_4af5_ba68_6627bc8465ec.slice/crio-42bc5ba764ae445f7fdd5a4b6316d35f7276c7150386cc3534f154c26f5e9b81 WatchSource:0}: Error finding container 42bc5ba764ae445f7fdd5a4b6316d35f7276c7150386cc3534f154c26f5e9b81: Status 404 returned error can't find the container with id 42bc5ba764ae445f7fdd5a4b6316d35f7276c7150386cc3534f154c26f5e9b81
Oct 30 21:43:14.594662 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:14.594574    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx" event={"ID":"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec","Type":"ContainerStarted","Data":"a22b96f79e09eb9596e20b192836b3f0da2cc8b56784a4683f712546dd8db282"}
Oct 30 21:43:14.594662 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:14.594632    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx" event={"ID":"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec","Type":"ContainerStarted","Data":"42bc5ba764ae445f7fdd5a4b6316d35f7276c7150386cc3534f154c26f5e9b81"}
Oct 30 21:43:14.726451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:14.726411    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx" podStartSLOduration=1.726397502 podStartE2EDuration="1.726397502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:14.72208234 +0000 UTC m=+1850.593201458" watchObservedRunningTime="2026-10-30 21:43:14.726397502 +0000 UTC m=+1850.597516619"
Oct 30 21:43:15.602808 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:15.602770    5786 generic.go:356] "Generic (PLEG): container finished" podID="79cbc4ba-4b7c-4af5-ba68-6627bc8465ec" containerID="a22b96f79e09eb9596e20b192836b3f0da2cc8b56784a4683f712546dd8db282" exitCode=0
Oct 30 21:43:15.603178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:15.602811    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx" event={"ID":"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec","Type":"ContainerDied","Data":"a22b96f79e09eb9596e20b192836b3f0da2cc8b56784a4683f712546dd8db282"}
Oct 30 21:43:16.784237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.784196    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:16.863616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.863592    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-xks-ca\") pod \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") "
Oct 30 21:43:16.863743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.863637    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-kube-api-access-p7v7f\" (UniqueName: \"kubernetes.io/projected/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-kube-api-access-p7v7f\") pod \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") "
Oct 30 21:43:16.863867 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.863849    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-openshift-ca\") pod \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\" (UID: \"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec\") "
Oct 30 21:43:16.864157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.864129    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-openshift-ca" pod "79cbc4ba-4b7c-4af5-ba68-6627bc8465ec" (UID: "79cbc4ba-4b7c-4af5-ba68-6627bc8465ec"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:16.864425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.864406    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:16.866278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.866255    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-xks-ca" pod "79cbc4ba-4b7c-4af5-ba68-6627bc8465ec" (UID: "79cbc4ba-4b7c-4af5-ba68-6627bc8465ec"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:16.866367 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.866275    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-kube-api-access-p7v7f" pod "79cbc4ba-4b7c-4af5-ba68-6627bc8465ec" (UID: "79cbc4ba-4b7c-4af5-ba68-6627bc8465ec"). InnerVolumeSpecName "kube-api-access-p7v7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:16.966345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.966309    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:16.966345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:16.966344    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7v7f\" (UniqueName: \"kubernetes.io/projected/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec-kube-api-access-p7v7f\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:17.628003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:17.627977    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"
Oct 30 21:43:17.628145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:17.628015    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx" event={"ID":"79cbc4ba-4b7c-4af5-ba68-6627bc8465ec","Type":"ContainerDied","Data":"42bc5ba764ae445f7fdd5a4b6316d35f7276c7150386cc3534f154c26f5e9b81"}
Oct 30 21:43:17.628145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:17.628060    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42bc5ba764ae445f7fdd5a4b6316d35f7276c7150386cc3534f154c26f5e9b81"
Oct 30 21:43:22.356687 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:22.356649    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:43:22.754145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.754117    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"]
Oct 30 21:43:22.754916 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.754894    5786 cpu_manager.go:414] "containerMap: removing container" podUID="79cbc4ba-4b7c-4af5-ba68-6627bc8465ec" containerName="revoke-keys"
Oct 30 21:43:22.755035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.754917    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79cbc4ba-4b7c-4af5-ba68-6627bc8465ec" containerName="revoke-keys"
Oct 30 21:43:22.755094 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.755076    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="79cbc4ba-4b7c-4af5-ba68-6627bc8465ec" containerName="revoke-keys"
Oct 30 21:43:22.760046 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.760026    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:22.827115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.827084    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"]
Oct 30 21:43:22.831529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.831506    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:22.831663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.831637    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndpzv\" (UniqueName: \"kubernetes.io/projected/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-kube-api-access-ndpzv\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:22.831731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.831699    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:22.831947 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.831928    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:22.872396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.872375    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:43:22.874663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.874642    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:43:22.876028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.876008    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"]
Oct 30 21:43:22.925648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.925626    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"]
Oct 30 21:43:22.932684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.932661    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:22.932897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.932872    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:22.932997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.932979    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:22.933052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.933035    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndpzv\" (UniqueName: \"kubernetes.io/projected/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-kube-api-access-ndpzv\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:22.933156 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.933141    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:22.933262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.933245    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrjrb\" (UniqueName: \"kubernetes.io/projected/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-kube-api-access-zrjrb\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:22.933602 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.933575    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:22.935688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:22.935666    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:23.002238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.002177    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndpzv\" (UniqueName: \"kubernetes.io/projected/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-kube-api-access-ndpzv\") pod \"maas-api-revoke-sub-3a0f3808-zkbks\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:23.036317 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.034306    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrjrb\" (UniqueName: \"kubernetes.io/projected/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-kube-api-access-zrjrb\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:23.036622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.036595    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:23.037100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.037062    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:23.037869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.037831    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:23.039694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.039674    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:23.073765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.073743    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:23.108396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.108376    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrjrb\" (UniqueName: \"kubernetes.io/projected/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-kube-api-access-zrjrb\") pod \"maas-api-revoke-sub-b89a9b36-7wcdl\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:23.146942 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.146668    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:23.200872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.199417    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"]
Oct 30 21:43:23.224565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.224535    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.314352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.296540    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-fe5385-openshift-default-dockercfg-jw2xs\""
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.315801    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-socket\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.315903    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-certs\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.315948    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa8f6777-2d13-4940-b11a-4083c144496d-istio-podinfo\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.316033    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa8f6777-2d13-4940-b11a-4083c144496d-istiod-ca-cert\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.316081    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-credential-socket\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.316113    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-data\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.316187    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-envoy\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.316292    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm8nb\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-kube-api-access-gm8nb\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.334525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.316342    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-istio-token\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.521489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521173    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa8f6777-2d13-4940-b11a-4083c144496d-istio-podinfo\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.521489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521299    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa8f6777-2d13-4940-b11a-4083c144496d-istiod-ca-cert\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.521489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521348    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-credential-socket\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.521489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521383    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-data\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.521489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521451    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-envoy\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.522294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521548    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm8nb\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-kube-api-access-gm8nb\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.522294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521596    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-istio-token\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.522294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521737    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-socket\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.522294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521834    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-certs\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.522294 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.521841    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"]
Oct 30 21:43:23.522573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.522394    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-certs\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.528073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.528046    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"]
Oct 30 21:43:23.530641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.530180    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa8f6777-2d13-4940-b11a-4083c144496d-istiod-ca-cert\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.530641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.530459    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-credential-socket\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.530868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.530681    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-data\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.532864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.532848    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-envoy\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.535275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.533711    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-socket\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.539562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.539528    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa8f6777-2d13-4940-b11a-4083c144496d-istio-podinfo\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.592631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.592599    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-istio-token\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.593274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.593243    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm8nb\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-kube-api-access-gm8nb\") pod \"e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") " pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.670885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.670854    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks" event={"ID":"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210","Type":"ContainerStarted","Data":"8bff498983a459c93be9a67bb767c13b93e4b7a22c1fa50495fcec378cf51037"}
Oct 30 21:43:23.875295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.873687    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:23.923562 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:23.923526    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"]
Oct 30 21:43:24.384005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.383978    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"]
Oct 30 21:43:24.389352 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:24.389321    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa8f6777_2d13_4940_b11a_4083c144496d.slice/crio-4fff7f14748b0d35e4938dcca9357efb5f26c73effad4911bd2f737e1f4d1bf1 WatchSource:0}: Error finding container 4fff7f14748b0d35e4938dcca9357efb5f26c73effad4911bd2f737e1f4d1bf1: Status 404 returned error can't find the container with id 4fff7f14748b0d35e4938dcca9357efb5f26c73effad4911bd2f737e1f4d1bf1
Oct 30 21:43:24.419444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.411227    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:43:24.419444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.411318    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:43:24.419444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.411361    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:43:24.679503 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.679424    5786 generic.go:356] "Generic (PLEG): container finished" podID="ec52b3ed-6a2a-4d0f-8e5b-639f54cac210" containerID="da6c6ef444aa2cd4f73ecea1a91a79dc7117c0e04c19316e39823c4dd74be259" exitCode=0
Oct 30 21:43:24.679891 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.679495    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks" event={"ID":"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210","Type":"ContainerDied","Data":"da6c6ef444aa2cd4f73ecea1a91a79dc7117c0e04c19316e39823c4dd74be259"}
Oct 30 21:43:24.681135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.681109    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl" event={"ID":"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2","Type":"ContainerStarted","Data":"4f9a607fb5da8fa7caaa64e1589a000fe5cb6967c33eb6da8d3d4fd90d6a229f"}
Oct 30 21:43:24.681275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.681157    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl" event={"ID":"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2","Type":"ContainerStarted","Data":"216914c5df497c65246bb82843b6153022c4df67cfcdafd79a1afe40ee4877fc"}
Oct 30 21:43:24.682542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.682517    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" event={"ID":"aa8f6777-2d13-4940-b11a-4083c144496d","Type":"ContainerStarted","Data":"4fff7f14748b0d35e4938dcca9357efb5f26c73effad4911bd2f737e1f4d1bf1"}
Oct 30 21:43:24.950426 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.950289    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl" podStartSLOduration=2.95027127 podStartE2EDuration="2.95027127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:24.94493045 +0000 UTC m=+1860.816049571" watchObservedRunningTime="2026-10-30 21:43:24.95027127 +0000 UTC m=+1860.821390391"
Oct 30 21:43:24.962389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:24.962360    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"]
Oct 30 21:43:25.070702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:25.070662    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"]
Oct 30 21:43:25.691798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:25.691768    5786 generic.go:356] "Generic (PLEG): container finished" podID="9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2" containerID="4f9a607fb5da8fa7caaa64e1589a000fe5cb6967c33eb6da8d3d4fd90d6a229f" exitCode=0
Oct 30 21:43:25.692298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:25.691862    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl" event={"ID":"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2","Type":"ContainerDied","Data":"4f9a607fb5da8fa7caaa64e1589a000fe5cb6967c33eb6da8d3d4fd90d6a229f"}
Oct 30 21:43:25.693687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:25.693662    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" event={"ID":"aa8f6777-2d13-4940-b11a-4083c144496d","Type":"ContainerStarted","Data":"efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995"}
Oct 30 21:43:25.856596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:25.856565    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:25.874776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:25.874754    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:25.880216 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:25.880180    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:25.930400 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:25.930352    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" podStartSLOduration=2.9303376500000002 podStartE2EDuration="2.93033765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:25.928653383 +0000 UTC m=+1861.799772520" watchObservedRunningTime="2026-10-30 21:43:25.93033765 +0000 UTC m=+1861.801456771"
Oct 30 21:43:26.021042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.020961    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-openshift-ca\") pod \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") "
Oct 30 21:43:26.021042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.021024    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-kube-api-access-ndpzv\" (UniqueName: \"kubernetes.io/projected/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-kube-api-access-ndpzv\") pod \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") "
Oct 30 21:43:26.021291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.021096    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-xks-ca\") pod \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\" (UID: \"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210\") "
Oct 30 21:43:26.021398 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.021375    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-openshift-ca" pod "ec52b3ed-6a2a-4d0f-8e5b-639f54cac210" (UID: "ec52b3ed-6a2a-4d0f-8e5b-639f54cac210"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:26.021789 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.021767    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:26.023247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.023221    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-xks-ca" pod "ec52b3ed-6a2a-4d0f-8e5b-639f54cac210" (UID: "ec52b3ed-6a2a-4d0f-8e5b-639f54cac210"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:26.023247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.023233    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-kube-api-access-ndpzv" pod "ec52b3ed-6a2a-4d0f-8e5b-639f54cac210" (UID: "ec52b3ed-6a2a-4d0f-8e5b-639f54cac210"). InnerVolumeSpecName "kube-api-access-ndpzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:26.131453 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.131418    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndpzv\" (UniqueName: \"kubernetes.io/projected/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-kube-api-access-ndpzv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:26.131453 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.131455    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:26.780816 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.779841    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"
Oct 30 21:43:26.781307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.781000    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks" event={"ID":"ec52b3ed-6a2a-4d0f-8e5b-639f54cac210","Type":"ContainerDied","Data":"8bff498983a459c93be9a67bb767c13b93e4b7a22c1fa50495fcec378cf51037"}
Oct 30 21:43:26.781307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.781048    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bff498983a459c93be9a67bb767c13b93e4b7a22c1fa50495fcec378cf51037"
Oct 30 21:43:26.812231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.802279    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:26.833228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:26.825472    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:43:27.159647 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.159614    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:27.194176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.194150    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-openshift-ca\") pod \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") "
Oct 30 21:43:27.194341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.194305    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-xks-ca\") pod \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") "
Oct 30 21:43:27.194612 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.194587    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-openshift-ca" pod "9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2" (UID: "9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:27.194942 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.194924    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:27.196848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.196823    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-xks-ca" pod "9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2" (UID: "9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:27.295828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.295749    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-kube-api-access-zrjrb\" (UniqueName: \"kubernetes.io/projected/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-kube-api-access-zrjrb\") pod \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\" (UID: \"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2\") "
Oct 30 21:43:27.296720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.296690    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:27.298427 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.298402    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-kube-api-access-zrjrb" pod "9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2" (UID: "9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2"). InnerVolumeSpecName "kube-api-access-zrjrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:27.397878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.397841    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrjrb\" (UniqueName: \"kubernetes.io/projected/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2-kube-api-access-zrjrb\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:27.788799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.788771    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"
Oct 30 21:43:27.789184 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.788797    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl" event={"ID":"9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2","Type":"ContainerDied","Data":"216914c5df497c65246bb82843b6153022c4df67cfcdafd79a1afe40ee4877fc"}
Oct 30 21:43:27.789184 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:27.788841    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="216914c5df497c65246bb82843b6153022c4df67cfcdafd79a1afe40ee4877fc"
Oct 30 21:43:28.443962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.443931    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"]
Oct 30 21:43:28.444237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.444185    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k" podUID="2670cbed-0d37-4982-99e3-e50b3308c23a" containerName="maas-api" containerID="cri-o://70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad" gracePeriod=30
Oct 30 21:43:28.796377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.796347    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:43:28.799829 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.799807    5786 generic.go:356] "Generic (PLEG): container finished" podID="2670cbed-0d37-4982-99e3-e50b3308c23a" containerID="70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad" exitCode=0
Oct 30 21:43:28.799954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.799848    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"
Oct 30 21:43:28.799954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.799892    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k" event={"ID":"2670cbed-0d37-4982-99e3-e50b3308c23a","Type":"ContainerDied","Data":"70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad"}
Oct 30 21:43:28.799954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.799941    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k" event={"ID":"2670cbed-0d37-4982-99e3-e50b3308c23a","Type":"ContainerDied","Data":"2087cad8e1fe28e5a540ce59afe59f3a4d70fc775cb52d5f84962f8c070b1b00"}
Oct 30 21:43:28.800137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.799971    5786 scope.go:122] "RemoveContainer" containerID="70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad"
Oct 30 21:43:28.809847 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.809832    5786 scope.go:122] "RemoveContainer" containerID="70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad"
Oct 30 21:43:28.810145 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:28.810117    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad\": container with ID starting with 70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad not found: ID does not exist" containerID="70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad"
Oct 30 21:43:28.810219 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.810154    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad"} err="failed to get container status \"70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad\": rpc error: code = NotFound desc = could not find container \"70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad\": container with ID starting with 70f4c32a22ec8417b2e8cbda78d1fd2fb6afd91cdc98512053a25ca1b150dfad not found: ID does not exist"
Oct 30 21:43:28.934263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.934200    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-metrics-certs\") pod \"2670cbed-0d37-4982-99e3-e50b3308c23a\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") "
Oct 30 21:43:28.934455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.934431    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2670cbed-0d37-4982-99e3-e50b3308c23a-kube-api-access-vx2tr\" (UniqueName: \"kubernetes.io/projected/2670cbed-0d37-4982-99e3-e50b3308c23a-kube-api-access-vx2tr\") pod \"2670cbed-0d37-4982-99e3-e50b3308c23a\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") "
Oct 30 21:43:28.934574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.934558    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls\") pod \"2670cbed-0d37-4982-99e3-e50b3308c23a\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") "
Oct 30 21:43:28.934761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.934656    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2670cbed-0d37-4982-99e3-e50b3308c23a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2670cbed-0d37-4982-99e3-e50b3308c23a-openshift-service-ca\") pod \"2670cbed-0d37-4982-99e3-e50b3308c23a\" (UID: \"2670cbed-0d37-4982-99e3-e50b3308c23a\") "
Oct 30 21:43:28.937881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.937846    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-metrics-certs" pod "2670cbed-0d37-4982-99e3-e50b3308c23a" (UID: "2670cbed-0d37-4982-99e3-e50b3308c23a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:28.947272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.938700    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2670cbed-0d37-4982-99e3-e50b3308c23a-openshift-service-ca" pod "2670cbed-0d37-4982-99e3-e50b3308c23a" (UID: "2670cbed-0d37-4982-99e3-e50b3308c23a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:28.947527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.947504    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2670cbed-0d37-4982-99e3-e50b3308c23a-kube-api-access-vx2tr" pod "2670cbed-0d37-4982-99e3-e50b3308c23a" (UID: "2670cbed-0d37-4982-99e3-e50b3308c23a"). InnerVolumeSpecName "kube-api-access-vx2tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:28.978245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:28.959370    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls" pod "2670cbed-0d37-4982-99e3-e50b3308c23a" (UID: "2670cbed-0d37-4982-99e3-e50b3308c23a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:29.046151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:29.046052    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx2tr\" (UniqueName: \"kubernetes.io/projected/2670cbed-0d37-4982-99e3-e50b3308c23a-kube-api-access-vx2tr\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:29.046151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:29.046089    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:29.046151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:29.046098    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2670cbed-0d37-4982-99e3-e50b3308c23a-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:29.046151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:29.046108    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2670cbed-0d37-4982-99e3-e50b3308c23a-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:29.305286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:29.292295    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"]
Oct 30 21:43:29.305286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:29.297889    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bf1b02c7-dbd749f89-pqt7k"]
Oct 30 21:43:29.962854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:29.962819    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq" podUID="b6bb0a20-dd91-4a81-9d6e-e8db064517af" containerName="payload-processing" containerID="cri-o://97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3" gracePeriod=25
Oct 30 21:43:30.071791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.071734    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq" podUID="07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3" containerName="payload-pre-processing" containerID="cri-o://93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967" gracePeriod=25
Oct 30 21:43:30.244879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.244848    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:43:30.312335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.312311    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6bb0a20-dd91-4a81-9d6e-e8db064517af-kube-api-access-m2t76\" (UniqueName: \"kubernetes.io/projected/b6bb0a20-dd91-4a81-9d6e-e8db064517af-kube-api-access-m2t76\") pod \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\" (UID: \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\") "
Oct 30 21:43:30.312581 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.312565    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6bb0a20-dd91-4a81-9d6e-e8db064517af-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6bb0a20-dd91-4a81-9d6e-e8db064517af-plugins-config-volume\") pod \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\" (UID: \"b6bb0a20-dd91-4a81-9d6e-e8db064517af\") "
Oct 30 21:43:30.312976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.312952    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6bb0a20-dd91-4a81-9d6e-e8db064517af-plugins-config-volume" pod "b6bb0a20-dd91-4a81-9d6e-e8db064517af" (UID: "b6bb0a20-dd91-4a81-9d6e-e8db064517af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:30.313166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.313154    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6bb0a20-dd91-4a81-9d6e-e8db064517af-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:30.315639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.315613    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6bb0a20-dd91-4a81-9d6e-e8db064517af-kube-api-access-m2t76" pod "b6bb0a20-dd91-4a81-9d6e-e8db064517af" (UID: "b6bb0a20-dd91-4a81-9d6e-e8db064517af"). InnerVolumeSpecName "kube-api-access-m2t76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:30.344975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.344953    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:43:30.360350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.360326    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2670cbed-0d37-4982-99e3-e50b3308c23a" path="/var/lib/kubelet/pods/2670cbed-0d37-4982-99e3-e50b3308c23a/volumes"
Oct 30 21:43:30.413706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.413680    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-kube-api-access-dw885\" (UniqueName: \"kubernetes.io/projected/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-kube-api-access-dw885\") pod \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\" (UID: \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\") "
Oct 30 21:43:30.413852 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.413784    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-plugins-config-volume\") pod \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\" (UID: \"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3\") "
Oct 30 21:43:30.414225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.414183    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-plugins-config-volume" pod "07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3" (UID: "07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:30.414577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.414554    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2t76\" (UniqueName: \"kubernetes.io/projected/b6bb0a20-dd91-4a81-9d6e-e8db064517af-kube-api-access-m2t76\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:30.414677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.414583    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:30.416066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.416043    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-kube-api-access-dw885" pod "07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3" (UID: "07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3"). InnerVolumeSpecName "kube-api-access-dw885". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:30.515459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.515391    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw885\" (UniqueName: \"kubernetes.io/projected/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3-kube-api-access-dw885\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:30.816996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.816908    5786 generic.go:356] "Generic (PLEG): container finished" podID="07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3" containerID="93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967" exitCode=0
Oct 30 21:43:30.816996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.816970    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"
Oct 30 21:43:30.817269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.817004    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq" event={"ID":"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3","Type":"ContainerDied","Data":"93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967"}
Oct 30 21:43:30.817269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.817066    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq" event={"ID":"07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3","Type":"ContainerDied","Data":"6c893381731a676005c5073d60a02070a7e617a1063a4496e028a466840fbdf3"}
Oct 30 21:43:30.817269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.817099    5786 scope.go:122] "RemoveContainer" containerID="93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967"
Oct 30 21:43:30.818766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.818743    5786 generic.go:356] "Generic (PLEG): container finished" podID="b6bb0a20-dd91-4a81-9d6e-e8db064517af" containerID="97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3" exitCode=0
Oct 30 21:43:30.818846 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.818794    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"
Oct 30 21:43:30.819004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.818931    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq" event={"ID":"b6bb0a20-dd91-4a81-9d6e-e8db064517af","Type":"ContainerDied","Data":"97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3"}
Oct 30 21:43:30.819004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.818975    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq" event={"ID":"b6bb0a20-dd91-4a81-9d6e-e8db064517af","Type":"ContainerDied","Data":"efe65411bb6d6d03237b72795a77b63a0ea8084c63a2dc54b61ae5c0cb480681"}
Oct 30 21:43:30.828625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.828611    5786 scope.go:122] "RemoveContainer" containerID="93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967"
Oct 30 21:43:30.828879 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:30.828861    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967\": container with ID starting with 93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967 not found: ID does not exist" containerID="93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967"
Oct 30 21:43:30.828925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.828886    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967"} err="failed to get container status \"93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967\": rpc error: code = NotFound desc = could not find container \"93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967\": container with ID starting with 93478e3189bbe086a4212075b009201737bc6bb19b3b30f05ef0df9329fac967 not found: ID does not exist"
Oct 30 21:43:30.828925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.828902    5786 scope.go:122] "RemoveContainer" containerID="97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3"
Oct 30 21:43:30.837813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.837794    5786 scope.go:122] "RemoveContainer" containerID="97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3"
Oct 30 21:43:30.838055 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:30.838037    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3\": container with ID starting with 97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3 not found: ID does not exist" containerID="97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3"
Oct 30 21:43:30.838125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.838063    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3"} err="failed to get container status \"97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3\": rpc error: code = NotFound desc = could not find container \"97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3\": container with ID starting with 97c999858f8accfa7ef9a259fde736b2b70855fe84dd93f12424effe061e04c3 not found: ID does not exist"
Oct 30 21:43:30.920717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.920684    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"]
Oct 30 21:43:30.939345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:30.939317    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf1b02c7-8xfhx"]
Oct 30 21:43:31.014973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:31.014948    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"]
Oct 30 21:43:31.018301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:31.018278    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bf1b02c7-55d6d6fcc9-bphlq"]
Oct 30 21:43:31.173858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:31.173833    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"]
Oct 30 21:43:31.195706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:31.195672    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bf1b02c7-686dfc467c-2wbzq"]
Oct 30 21:43:32.360927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:32.360894    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3" path="/var/lib/kubelet/pods/07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3/volumes"
Oct 30 21:43:32.361380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:32.361365    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79cbc4ba-4b7c-4af5-ba68-6627bc8465ec" path="/var/lib/kubelet/pods/79cbc4ba-4b7c-4af5-ba68-6627bc8465ec/volumes"
Oct 30 21:43:32.361753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:32.361740    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6bb0a20-dd91-4a81-9d6e-e8db064517af" path="/var/lib/kubelet/pods/b6bb0a20-dd91-4a81-9d6e-e8db064517af/volumes"
Oct 30 21:43:33.647122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.647089    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"]
Oct 30 21:43:33.647993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.647970    5786 cpu_manager.go:414] "containerMap: removing container" podUID="2670cbed-0d37-4982-99e3-e50b3308c23a" containerName="maas-api"
Oct 30 21:43:33.648098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.647995    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2670cbed-0d37-4982-99e3-e50b3308c23a" containerName="maas-api"
Oct 30 21:43:33.648098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648034    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b6bb0a20-dd91-4a81-9d6e-e8db064517af" containerName="payload-processing"
Oct 30 21:43:33.648098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648043    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6bb0a20-dd91-4a81-9d6e-e8db064517af" containerName="payload-processing"
Oct 30 21:43:33.648098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648057    5786 cpu_manager.go:414] "containerMap: removing container" podUID="9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2" containerName="revoke-keys"
Oct 30 21:43:33.648098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648065    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2" containerName="revoke-keys"
Oct 30 21:43:33.648381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648099    5786 cpu_manager.go:414] "containerMap: removing container" podUID="07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3" containerName="payload-pre-processing"
Oct 30 21:43:33.648381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648106    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3" containerName="payload-pre-processing"
Oct 30 21:43:33.648381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648121    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ec52b3ed-6a2a-4d0f-8e5b-639f54cac210" containerName="revoke-keys"
Oct 30 21:43:33.648381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648127    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec52b3ed-6a2a-4d0f-8e5b-639f54cac210" containerName="revoke-keys"
Oct 30 21:43:33.648381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648329    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="2670cbed-0d37-4982-99e3-e50b3308c23a" containerName="maas-api"
Oct 30 21:43:33.648381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648350    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6bb0a20-dd91-4a81-9d6e-e8db064517af" containerName="payload-processing"
Oct 30 21:43:33.648381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648361    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2" containerName="revoke-keys"
Oct 30 21:43:33.648381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648377    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="07686cd0-e97e-43bf-9dc6-f73d0f3a1ee3" containerName="payload-pre-processing"
Oct 30 21:43:33.648736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.648387    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec52b3ed-6a2a-4d0f-8e5b-639f54cac210" containerName="revoke-keys"
Oct 30 21:43:33.653642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.653608    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:33.670751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.670725    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7\" (UID: \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:33.670874 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.670761    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjnx6\" (UniqueName: \"kubernetes.io/projected/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-kube-api-access-kjnx6\") pod \"payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7\" (UID: \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:33.727033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.726993    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"]
Oct 30 21:43:33.769332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.769300    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-fe5385\""
Oct 30 21:43:33.769919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.769849    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-fe5385-dockercfg-5bvsz\""
Oct 30 21:43:33.779161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.779122    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7\" (UID: \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:33.779314 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.779177    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjnx6\" (UniqueName: \"kubernetes.io/projected/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-kube-api-access-kjnx6\") pod \"payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7\" (UID: \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:33.781516 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.780870    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7\" (UID: \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:33.848673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.848644    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjnx6\" (UniqueName: \"kubernetes.io/projected/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-kube-api-access-kjnx6\") pod \"payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7\" (UID: \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:33.889153 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.889114    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"]
Oct 30 21:43:33.902957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.902888    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:33.968664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.968641    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:33.976753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.976730    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"]
Oct 30 21:43:33.988646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.988601    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lggls\" (UniqueName: \"kubernetes.io/projected/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-kube-api-access-lggls\") pod \"payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt\" (UID: \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:33.988878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:33.988826    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt\" (UID: \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:34.090919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.090881    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lggls\" (UniqueName: \"kubernetes.io/projected/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-kube-api-access-lggls\") pod \"payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt\" (UID: \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:34.091078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.091016    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt\" (UID: \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:34.092799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.092771    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt\" (UID: \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:34.150103 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.150079    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lggls\" (UniqueName: \"kubernetes.io/projected/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-kube-api-access-lggls\") pod \"payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt\" (UID: \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:34.224282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.224258    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:34.378725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.378699    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"]
Oct 30 21:43:34.379747 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:34.379722    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0116a4e_e09d_4d5a_9948_369d7cbb17ea.slice/crio-6078b0d206168f3741efe2b5fe7dca47fdd6ea23b2e517d9cd1abdda30c971b0 WatchSource:0}: Error finding container 6078b0d206168f3741efe2b5fe7dca47fdd6ea23b2e517d9cd1abdda30c971b0: Status 404 returned error can't find the container with id 6078b0d206168f3741efe2b5fe7dca47fdd6ea23b2e517d9cd1abdda30c971b0
Oct 30 21:43:34.609383 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.609357    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"]
Oct 30 21:43:34.610303 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:34.610275    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b9c5cd_2e9b_4cec_8499_17c84cd36401.slice/crio-9cb990ce8b1687c6a54da98d2fc946a9472bf3d2cfdf783156f0bd2ae81ad188 WatchSource:0}: Error finding container 9cb990ce8b1687c6a54da98d2fc946a9472bf3d2cfdf783156f0bd2ae81ad188: Status 404 returned error can't find the container with id 9cb990ce8b1687c6a54da98d2fc946a9472bf3d2cfdf783156f0bd2ae81ad188
Oct 30 21:43:34.877967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.877889    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt" event={"ID":"e6b9c5cd-2e9b-4cec-8499-17c84cd36401","Type":"ContainerStarted","Data":"408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a"}
Oct 30 21:43:34.877967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.877945    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt" event={"ID":"e6b9c5cd-2e9b-4cec-8499-17c84cd36401","Type":"ContainerStarted","Data":"9cb990ce8b1687c6a54da98d2fc946a9472bf3d2cfdf783156f0bd2ae81ad188"}
Oct 30 21:43:34.878425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.877972    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:34.879648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.879624    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7" event={"ID":"b0116a4e-e09d-4d5a-9948-369d7cbb17ea","Type":"ContainerStarted","Data":"2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9"}
Oct 30 21:43:34.879751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.879664    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7" event={"ID":"b0116a4e-e09d-4d5a-9948-369d7cbb17ea","Type":"ContainerStarted","Data":"6078b0d206168f3741efe2b5fe7dca47fdd6ea23b2e517d9cd1abdda30c971b0"}
Oct 30 21:43:34.879751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:34.879700    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:35.008253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:35.008199    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt" podStartSLOduration=2.008186565 podStartE2EDuration="2.008186565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:35.004508254 +0000 UTC m=+1870.875627374" watchObservedRunningTime="2026-10-30 21:43:35.008186565 +0000 UTC m=+1870.879305681"
Oct 30 21:43:35.166290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:35.165034    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7" podStartSLOduration=2.165011828 podStartE2EDuration="2.165011828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:35.161606359 +0000 UTC m=+1871.032725527" watchObservedRunningTime="2026-10-30 21:43:35.165011828 +0000 UTC m=+1871.036130957"
Oct 30 21:43:36.215283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.215250    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"]
Oct 30 21:43:36.219723 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.219706    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.320134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.320095    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c025712-f515-4f27-a581-1b9e85f3ee04-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.320345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.320244    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.320438 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.320360    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.320438 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.320388    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2p6d\" (UniqueName: \"kubernetes.io/projected/4c025712-f515-4f27-a581-1b9e85f3ee04-kube-api-access-w2p6d\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.324735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.324715    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-5f31bd-serving-cert\""
Oct 30 21:43:36.328347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.328330    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"]
Oct 30 21:43:36.421966 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.421932    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c025712-f515-4f27-a581-1b9e85f3ee04-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.422145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.422004    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.422145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.422085    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.422145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.422110    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2p6d\" (UniqueName: \"kubernetes.io/projected/4c025712-f515-4f27-a581-1b9e85f3ee04-kube-api-access-w2p6d\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.422385 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:36.422261    5786 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-serving-cert: secret "maas-api-e2e-shared-b-w1-5f31bd-serving-cert" not found
Oct 30 21:43:36.422385 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:36.422330    5786 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls podName:4c025712-f515-4f27-a581-1b9e85f3ee04 nodeName:}" failed. No retries permitted until 2026-10-30 21:43:36.922311721 +0000 UTC m=+1872.793430832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls") pod "maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb" (UID: "4c025712-f515-4f27-a581-1b9e85f3ee04") : secret "maas-api-e2e-shared-b-w1-5f31bd-serving-cert" not found
Oct 30 21:43:36.422687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.422665    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c025712-f515-4f27-a581-1b9e85f3ee04-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.424604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.424586    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.521273 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.520583    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2p6d\" (UniqueName: \"kubernetes.io/projected/4c025712-f515-4f27-a581-1b9e85f3ee04-kube-api-access-w2p6d\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.943518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.943487    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:36.946131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:36.946107    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:37.134238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:37.134197    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:37.528780 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:37.528699    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c025712_f515_4f27_a581_1b9e85f3ee04.slice/crio-41819e2b67df90122122c7c787c2768d1e06f13c012c54e52fedda2f0a8adfea WatchSource:0}: Error finding container 41819e2b67df90122122c7c787c2768d1e06f13c012c54e52fedda2f0a8adfea: Status 404 returned error can't find the container with id 41819e2b67df90122122c7c787c2768d1e06f13c012c54e52fedda2f0a8adfea
Oct 30 21:43:37.533370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:37.533345    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"]
Oct 30 21:43:37.534297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:37.534278    5786 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:43:37.903126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:37.903052    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb" event={"ID":"4c025712-f515-4f27-a581-1b9e85f3ee04","Type":"ContainerStarted","Data":"41819e2b67df90122122c7c787c2768d1e06f13c012c54e52fedda2f0a8adfea"}
Oct 30 21:43:38.912247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:38.912184    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb" event={"ID":"4c025712-f515-4f27-a581-1b9e85f3ee04","Type":"ContainerStarted","Data":"6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a"}
Oct 30 21:43:38.912669 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:38.912301    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:40.514612 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:40.514559    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb" podStartSLOduration=3.680494875 podStartE2EDuration="4.514540611s" totalImagesPullingTime="834.045736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:39.059508031 +0000 UTC m=+1874.930627149" watchObservedRunningTime="2026-10-30 21:43:40.514540611 +0000 UTC m=+1876.385659736"
Oct 30 21:43:40.518451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:40.518428    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"]
Oct 30 21:43:40.518616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:40.518598    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy" containerID="cri-o://6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c" gracePeriod=30
Oct 30 21:43:44.928051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:44.928016    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:43:45.794093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.794039    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:43:45.889045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889023    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:43:45.889176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889086    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:43:45.889629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889576    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-certs\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.889740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889631    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed744068-8b59-4580-b1ec-e34a5f40a63e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed744068-8b59-4580-b1ec-e34a5f40a63e-istiod-ca-cert\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.889740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889657    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-envoy\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.889740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889704    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-podinfo\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.889740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889728    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-kube-api-access-4jdpz\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-kube-api-access-4jdpz\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.889972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889844    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-certs" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:43:45.889972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889867    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-token\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-token\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.889972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.889950    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-socket\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.890126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890039    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-credential-socket\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.890126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890059    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed744068-8b59-4580-b1ec-e34a5f40a63e-istiod-ca-cert" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:45.890126 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890087    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-data\") pod \"ed744068-8b59-4580-b1ec-e34a5f40a63e\" (UID: \"ed744068-8b59-4580-b1ec-e34a5f40a63e\") "
Oct 30 21:43:45.890550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890353    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-credential-socket" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:43:45.890550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890399    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-socket" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:43:45.890550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890516    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-data" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:43:45.890793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890608    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.890793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890622    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.890793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890635    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.890793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890644    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.890793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.890652    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed744068-8b59-4580-b1ec-e34a5f40a63e-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.892468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.892440    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-kube-api-access-4jdpz" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "kube-api-access-4jdpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:45.892568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.892476    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-envoy" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:43:45.892568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.892543    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-podinfo" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:43:45.892568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.892543    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-token" pod "ed744068-8b59-4580-b1ec-e34a5f40a63e" (UID: "ed744068-8b59-4580-b1ec-e34a5f40a63e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:45.970445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.970225    5786 generic.go:356] "Generic (PLEG): container finished" podID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerID="6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c" exitCode=0
Oct 30 21:43:45.970445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.970284    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"
Oct 30 21:43:45.970445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.970311    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" event={"ID":"ed744068-8b59-4580-b1ec-e34a5f40a63e","Type":"ContainerDied","Data":"6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c"}
Oct 30 21:43:45.970445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.970374    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5" event={"ID":"ed744068-8b59-4580-b1ec-e34a5f40a63e","Type":"ContainerDied","Data":"316dbdc32fe1b83598ba3cb2b8e078cc4f06559d06b135dc6b2793b8e57de062"}
Oct 30 21:43:45.970445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.970404    5786 scope.go:122] "RemoveContainer" containerID="6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c"
Oct 30 21:43:45.982978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.982961    5786 scope.go:122] "RemoveContainer" containerID="4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761"
Oct 30 21:43:45.992643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.992359    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.992643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.992387    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.992643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.992401    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed744068-8b59-4580-b1ec-e34a5f40a63e-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.992643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.992415    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jdpz\" (UniqueName: \"kubernetes.io/projected/ed744068-8b59-4580-b1ec-e34a5f40a63e-kube-api-access-4jdpz\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:45.994711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.994689    5786 scope.go:122] "RemoveContainer" containerID="6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c"
Oct 30 21:43:45.995007 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:45.994987    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c\": container with ID starting with 6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c not found: ID does not exist" containerID="6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c"
Oct 30 21:43:45.995107 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.995014    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c"} err="failed to get container status \"6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c\": rpc error: code = NotFound desc = could not find container \"6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c\": container with ID starting with 6ccc69fa8c1b5c56a7aa5addc1cb37e8ac717834fc1b3f8338a3ce95807c6e3c not found: ID does not exist"
Oct 30 21:43:45.995107 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.995031    5786 scope.go:122] "RemoveContainer" containerID="4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761"
Oct 30 21:43:45.995340 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:45.995309    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761\": container with ID starting with 4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761 not found: ID does not exist" containerID="4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761"
Oct 30 21:43:45.995428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:45.995345    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761"} err="failed to get container status \"4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761\": rpc error: code = NotFound desc = could not find container \"4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761\": container with ID starting with 4c15e99a711f6df34e7e29277d5e0166ce17df3b18838a6d0b63f7fcc743c761 not found: ID does not exist"
Oct 30 21:43:46.131567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.131498    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"]
Oct 30 21:43:46.366455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.366423    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"]
Oct 30 21:43:46.380152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.380111    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bf1b02c7-openshift-default-75465d69c-hljn5"]
Oct 30 21:43:46.507010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.506979    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"]
Oct 30 21:43:46.507238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.507183    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc" podUID="c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" containerName="maas-api" containerID="cri-o://b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b" gracePeriod=30
Oct 30 21:43:46.842005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.841981    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:43:46.965233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.964571    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-kube-api-access-6r2xv\" (UniqueName: \"kubernetes.io/projected/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-kube-api-access-6r2xv\") pod \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") "
Oct 30 21:43:46.965233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.964724    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-openshift-service-ca\") pod \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") "
Oct 30 21:43:46.965233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.964876    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-metrics-certs\") pod \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") "
Oct 30 21:43:46.965233 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.964975    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls\") pod \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\" (UID: \"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b\") "
Oct 30 21:43:46.977368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:46.977025    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-openshift-service-ca" pod "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" (UID: "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:47.008656 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.008346    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-metrics-certs" pod "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" (UID: "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:47.036120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.036090    5786 generic.go:356] "Generic (PLEG): container finished" podID="c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" containerID="b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b" exitCode=0
Oct 30 21:43:47.036353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.036332    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc" event={"ID":"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b","Type":"ContainerDied","Data":"b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b"}
Oct 30 21:43:47.036473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.036390    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc" event={"ID":"c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b","Type":"ContainerDied","Data":"73fea11041d0dcc491cafc14cc8487f478f041fb18e7d5db8fd2dbd8100dee00"}
Oct 30 21:43:47.036473 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.036423    5786 scope.go:122] "RemoveContainer" containerID="b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b"
Oct 30 21:43:47.036949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.036578    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"
Oct 30 21:43:47.049036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.049003    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls" pod "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" (UID: "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:47.049899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.049879    5786 scope.go:122] "RemoveContainer" containerID="b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b"
Oct 30 21:43:47.050328 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:47.050258    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b\": container with ID starting with b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b not found: ID does not exist" containerID="b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b"
Oct 30 21:43:47.050328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.050292    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b"} err="failed to get container status \"b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b\": rpc error: code = NotFound desc = could not find container \"b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b\": container with ID starting with b7c75175da1fee38cf384a2ec43ff1f9eba3b7bda2cf8f271fcd89cfc82c560b not found: ID does not exist"
Oct 30 21:43:47.050631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.050608    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-kube-api-access-6r2xv" pod "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" (UID: "c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b"). InnerVolumeSpecName "kube-api-access-6r2xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:47.067330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.067277    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.067330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.067305    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r2xv\" (UniqueName: \"kubernetes.io/projected/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-kube-api-access-6r2xv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.067330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.067319    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.067497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.067334    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.482231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.474441    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"]
Oct 30 21:43:47.528228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:47.527681    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9ac5c5-78f9db6857-rxjlc"]
Oct 30 21:43:48.360309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:48.360276    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" path="/var/lib/kubelet/pods/c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b/volumes"
Oct 30 21:43:48.360756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:48.360741    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" path="/var/lib/kubelet/pods/ed744068-8b59-4580-b1ec-e34a5f40a63e/volumes"
Oct 30 21:43:48.697377 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:48.697344    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"]
Oct 30 21:43:48.745689 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:48.745662    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9ac5c5-24vb6"]
Oct 30 21:43:50.360564 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:50.360529    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03dd650a-02ff-4244-9434-e5c658f80548" path="/var/lib/kubelet/pods/03dd650a-02ff-4244-9434-e5c658f80548/volumes"
Oct 30 21:43:51.628946 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.628911    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"]
Oct 30 21:43:51.629585 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629570    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy"
Oct 30 21:43:51.629642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629586    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy"
Oct 30 21:43:51.629642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629605    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy"
Oct 30 21:43:51.629642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629611    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy"
Oct 30 21:43:51.629642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629630    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" containerName="maas-api"
Oct 30 21:43:51.629642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629636    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" containerName="maas-api"
Oct 30 21:43:51.629848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629833    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy"
Oct 30 21:43:51.629892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629868    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed744068-8b59-4580-b1ec-e34a5f40a63e" containerName="istio-proxy"
Oct 30 21:43:51.629892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.629880    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0720946-3b9c-4a5e-b3c8-d3e99d5fa11b" containerName="maas-api"
Oct 30 21:43:51.635296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.635280    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.696156 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.696134    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"]
Oct 30 21:43:51.740493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.740470    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:43:51.740493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.740487    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:43:51.743765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.743745    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-894kw\" (UniqueName: \"kubernetes.io/projected/b22a470d-c6d8-4088-9f80-bf270cd5233d-kube-api-access-894kw\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.743842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.743795    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.743842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.743819    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.845188 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.845159    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-894kw\" (UniqueName: \"kubernetes.io/projected/b22a470d-c6d8-4088-9f80-bf270cd5233d-kube-api-access-894kw\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.845329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.845248    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.845329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.845272    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.845850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.845834    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.847676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.847660    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.903640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.903588    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-894kw\" (UniqueName: \"kubernetes.io/projected/b22a470d-c6d8-4088-9f80-bf270cd5233d-kube-api-access-894kw\") pod \"maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:51.948327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:51.948305    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:52.329908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:52.329877    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"]
Oct 30 21:43:52.330849 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:52.330815    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22a470d_c6d8_4088_9f80_bf270cd5233d.slice/crio-0c8243b1e904ea2016bf27251ec10bdc483b64b19affa1f27a623f8a84678ad1 WatchSource:0}: Error finding container 0c8243b1e904ea2016bf27251ec10bdc483b64b19affa1f27a623f8a84678ad1: Status 404 returned error can't find the container with id 0c8243b1e904ea2016bf27251ec10bdc483b64b19affa1f27a623f8a84678ad1
Oct 30 21:43:52.361298 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:52.361263    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:43:53.086661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.086629    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t" event={"ID":"b22a470d-c6d8-4088-9f80-bf270cd5233d","Type":"ContainerStarted","Data":"e7d847181151b3e1fde439214e9139970580de3473705213e4861b64a85dc0dd"}
Oct 30 21:43:53.087120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.086692    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t" event={"ID":"b22a470d-c6d8-4088-9f80-bf270cd5233d","Type":"ContainerStarted","Data":"0c8243b1e904ea2016bf27251ec10bdc483b64b19affa1f27a623f8a84678ad1"}
Oct 30 21:43:53.209783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.209739    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t" podStartSLOduration=2.209725239 podStartE2EDuration="2.209725239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:53.206933248 +0000 UTC m=+1889.078052368" watchObservedRunningTime="2026-10-30 21:43:53.209725239 +0000 UTC m=+1889.080844361"
Oct 30 21:43:53.434333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.434305    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"]
Oct 30 21:43:53.438543 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.438526    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.495947 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.495841    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-e0f967-serving-cert\""
Oct 30 21:43:53.500276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.500253    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"]
Oct 30 21:43:53.574594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.574556    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b8m4\" (UniqueName: \"kubernetes.io/projected/4bc20589-8be4-4048-a8ea-05d5a4b325ac-kube-api-access-5b8m4\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.574745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.574687    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc20589-8be4-4048-a8ea-05d5a4b325ac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.574813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.574797    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.574937 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.574916    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.676272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.676245    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b8m4\" (UniqueName: \"kubernetes.io/projected/4bc20589-8be4-4048-a8ea-05d5a4b325ac-kube-api-access-5b8m4\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.676433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.676329    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc20589-8be4-4048-a8ea-05d5a4b325ac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.676433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.676425    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.676552 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.676523    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.677002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.676980    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc20589-8be4-4048-a8ea-05d5a4b325ac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.678780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.678762    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.678935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.678917    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.740904 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.740881    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b8m4\" (UniqueName: \"kubernetes.io/projected/4bc20589-8be4-4048-a8ea-05d5a4b325ac-kube-api-access-5b8m4\") pod \"maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:53.752916 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:53.752895    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:54.095964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:54.095888    5786 generic.go:356] "Generic (PLEG): container finished" podID="b22a470d-c6d8-4088-9f80-bf270cd5233d" containerID="e7d847181151b3e1fde439214e9139970580de3473705213e4861b64a85dc0dd" exitCode=0
Oct 30 21:43:54.095964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:54.095921    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t" event={"ID":"b22a470d-c6d8-4088-9f80-bf270cd5233d","Type":"ContainerDied","Data":"e7d847181151b3e1fde439214e9139970580de3473705213e4861b64a85dc0dd"}
Oct 30 21:43:54.136261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:54.136238    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"]
Oct 30 21:43:54.137285 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:43:54.137260    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc20589_8be4_4048_a8ea_05d5a4b325ac.slice/crio-6969126c4ddac41e14f1322fe2b3c0ffdf297e9c2177ffbc4e8ce1eea4259246 WatchSource:0}: Error finding container 6969126c4ddac41e14f1322fe2b3c0ffdf297e9c2177ffbc4e8ce1eea4259246: Status 404 returned error can't find the container with id 6969126c4ddac41e14f1322fe2b3c0ffdf297e9c2177ffbc4e8ce1eea4259246
Oct 30 21:43:55.105259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.105226    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6" event={"ID":"4bc20589-8be4-4048-a8ea-05d5a4b325ac","Type":"ContainerStarted","Data":"6969126c4ddac41e14f1322fe2b3c0ffdf297e9c2177ffbc4e8ce1eea4259246"}
Oct 30 21:43:55.297161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.297109    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:55.411057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.411028    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b22a470d-c6d8-4088-9f80-bf270cd5233d-kube-api-access-894kw\" (UniqueName: \"kubernetes.io/projected/b22a470d-c6d8-4088-9f80-bf270cd5233d-kube-api-access-894kw\") pod \"b22a470d-c6d8-4088-9f80-bf270cd5233d\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") "
Oct 30 21:43:55.411180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.411132    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-openshift-ca\") pod \"b22a470d-c6d8-4088-9f80-bf270cd5233d\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") "
Oct 30 21:43:55.411352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.411333    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-xks-ca\") pod \"b22a470d-c6d8-4088-9f80-bf270cd5233d\" (UID: \"b22a470d-c6d8-4088-9f80-bf270cd5233d\") "
Oct 30 21:43:55.411472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.411441    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-openshift-ca" pod "b22a470d-c6d8-4088-9f80-bf270cd5233d" (UID: "b22a470d-c6d8-4088-9f80-bf270cd5233d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:55.411881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.411828    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:55.413478 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.413434    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b22a470d-c6d8-4088-9f80-bf270cd5233d-kube-api-access-894kw" pod "b22a470d-c6d8-4088-9f80-bf270cd5233d" (UID: "b22a470d-c6d8-4088-9f80-bf270cd5233d"). InnerVolumeSpecName "kube-api-access-894kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:55.413733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.413711    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-xks-ca" pod "b22a470d-c6d8-4088-9f80-bf270cd5233d" (UID: "b22a470d-c6d8-4088-9f80-bf270cd5233d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:55.513031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.513004    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-894kw\" (UniqueName: \"kubernetes.io/projected/b22a470d-c6d8-4088-9f80-bf270cd5233d-kube-api-access-894kw\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:55.513031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:55.513031    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b22a470d-c6d8-4088-9f80-bf270cd5233d-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:56.113929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:56.113900    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"
Oct 30 21:43:56.114375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:56.113940    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t" event={"ID":"b22a470d-c6d8-4088-9f80-bf270cd5233d","Type":"ContainerDied","Data":"0c8243b1e904ea2016bf27251ec10bdc483b64b19affa1f27a623f8a84678ad1"}
Oct 30 21:43:56.114375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:56.113991    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c8243b1e904ea2016bf27251ec10bdc483b64b19affa1f27a623f8a84678ad1"
Oct 30 21:43:56.115779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:56.115754    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6" event={"ID":"4bc20589-8be4-4048-a8ea-05d5a4b325ac","Type":"ContainerStarted","Data":"4de71047b91848e101d119428db6f0cf8bf8179b7d25f47e6857d7a036cb1ac5"}
Oct 30 21:43:56.115901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:56.115882    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:43:56.235201 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:56.235161    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6" podStartSLOduration=2.336164039 podStartE2EDuration="3.23514903s" totalImagesPullingTime="898.984991ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:56.231413676 +0000 UTC m=+1892.102532798" watchObservedRunningTime="2026-10-30 21:43:56.23514903 +0000 UTC m=+1892.106268149"
Oct 30 21:43:58.760595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.760565    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"]
Oct 30 21:43:58.761276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.761258    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b22a470d-c6d8-4088-9f80-bf270cd5233d" containerName="revoke-keys"
Oct 30 21:43:58.761326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.761278    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b22a470d-c6d8-4088-9f80-bf270cd5233d" containerName="revoke-keys"
Oct 30 21:43:58.761408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.761398    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b22a470d-c6d8-4088-9f80-bf270cd5233d" containerName="revoke-keys"
Oct 30 21:43:58.766353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.766334    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.820838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.820812    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"]
Oct 30 21:43:58.864861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.864836    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-99003c-serving-cert\""
Oct 30 21:43:58.865451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.865434    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm8h9\" (UniqueName: \"kubernetes.io/projected/c8d8457a-0a0c-4cdf-864a-cf5209912d34-kube-api-access-pm8h9\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.865518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.865502    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.865639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.865618    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d8457a-0a0c-4cdf-864a-cf5209912d34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.865689 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.865666    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.967265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.967183    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.967408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.967288    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d8457a-0a0c-4cdf-864a-cf5209912d34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.967408 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.967320    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.967504 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.967433    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm8h9\" (UniqueName: \"kubernetes.io/projected/c8d8457a-0a0c-4cdf-864a-cf5209912d34-kube-api-access-pm8h9\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.967583 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:58.967564    5786 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-serving-cert: secret "maas-api-e2e-shared-b-w2-99003c-serving-cert" not found
Oct 30 21:43:58.967650 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:43:58.967637    5786 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls podName:c8d8457a-0a0c-4cdf-864a-cf5209912d34 nodeName:}" failed. No retries permitted until 2026-10-30 21:43:59.467617053 +0000 UTC m=+1895.338736167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls") pod "maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp" (UID: "c8d8457a-0a0c-4cdf-864a-cf5209912d34") : secret "maas-api-e2e-shared-b-w2-99003c-serving-cert" not found
Oct 30 21:43:58.967949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.967930    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d8457a-0a0c-4cdf-864a-cf5209912d34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:58.969757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:58.969739    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:59.040276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:59.037136    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm8h9\" (UniqueName: \"kubernetes.io/projected/c8d8457a-0a0c-4cdf-864a-cf5209912d34-kube-api-access-pm8h9\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:59.476868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:59.476840    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:59.479358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:59.479330    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:43:59.679852 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:43:59.679829    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:44:00.058124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:00.058098    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"]
Oct 30 21:44:00.058751 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:44:00.058725    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d8457a_0a0c_4cdf_864a_cf5209912d34.slice/crio-7362e9b304c76b39060d1aaf6e8167d6f414c2e398abd4f6b5627af1c9ba894a WatchSource:0}: Error finding container 7362e9b304c76b39060d1aaf6e8167d6f414c2e398abd4f6b5627af1c9ba894a: Status 404 returned error can't find the container with id 7362e9b304c76b39060d1aaf6e8167d6f414c2e398abd4f6b5627af1c9ba894a
Oct 30 21:44:00.144982 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:00.144951    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp" event={"ID":"c8d8457a-0a0c-4cdf-864a-cf5209912d34","Type":"ContainerStarted","Data":"7362e9b304c76b39060d1aaf6e8167d6f414c2e398abd4f6b5627af1c9ba894a"}
Oct 30 21:44:01.154630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.154253    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main" containerID="cri-o://7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e" gracePeriod=45
Oct 30 21:44:01.154630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.154298    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp" event={"ID":"c8d8457a-0a0c-4cdf-864a-cf5209912d34","Type":"ContainerStarted","Data":"b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99"}
Oct 30 21:44:01.154630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.154388    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:44:01.489606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.489576    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:44:01.608523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.608447    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp" podStartSLOduration=2.7902251380000003 podStartE2EDuration="3.608428536s" totalImagesPullingTime="818.203398ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:01.288710694 +0000 UTC m=+1897.159829813" watchObservedRunningTime="2026-10-30 21:44:01.608428536 +0000 UTC m=+1897.479547656"
Oct 30 21:44:01.652988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.652961    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-dshm\") pod \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") "
Oct 30 21:44:01.653169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.652996    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-home\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-home\") pod \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") "
Oct 30 21:44:01.653169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.653165    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-model-cache\") pod \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") "
Oct 30 21:44:01.653330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.653304    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-kube-api-access-8qw4h\" (UniqueName: \"kubernetes.io/projected/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-kube-api-access-8qw4h\") pod \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") "
Oct 30 21:44:01.653391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.653339    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tmp-dir\") pod \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") "
Oct 30 21:44:01.653572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.653551    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-home" pod "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" (UID: "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:01.653572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.653560    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-model-cache" pod "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" (UID: "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:01.653831 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.653811    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tmp-dir" pod "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" (UID: "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:01.653911 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.653878    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tls-certs\" (UniqueName: \"kubernetes.io/secret/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tls-certs\") pod \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\" (UID: \"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c\") "
Oct 30 21:44:01.654850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.654824    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.655009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.654855    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.655009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.654870    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.656828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.656801    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-kube-api-access-8qw4h" pod "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" (UID: "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c"). InnerVolumeSpecName "kube-api-access-8qw4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:01.657281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.657252    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-dshm" pod "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" (UID: "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:01.662099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.662027    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tls-certs" pod "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" (UID: "2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:01.757012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.756936    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.757012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.756966    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qw4h\" (UniqueName: \"kubernetes.io/projected/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-kube-api-access-8qw4h\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.757012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:01.756975    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:02.138613 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.138518    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:44:02.164576 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.164547    5786 generic.go:356] "Generic (PLEG): container finished" podID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerID="7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e" exitCode=0
Oct 30 21:44:02.165028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.164594    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"
Oct 30 21:44:02.165028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.164604    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" event={"ID":"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c","Type":"ContainerDied","Data":"7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e"}
Oct 30 21:44:02.165028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.164655    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7" event={"ID":"2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c","Type":"ContainerDied","Data":"c7fc8a7bde500d4ec09081c5313708fdac44a9c3619ca506bac280bd6c3b77a7"}
Oct 30 21:44:02.165028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.164680    5786 scope.go:122] "RemoveContainer" containerID="7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e"
Oct 30 21:44:02.176032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.176014    5786 scope.go:122] "RemoveContainer" containerID="7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e"
Oct 30 21:44:02.176322 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:44:02.176303    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e\": container with ID starting with 7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e not found: ID does not exist" containerID="7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e"
Oct 30 21:44:02.176404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.176328    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e"} err="failed to get container status \"7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e\": rpc error: code = NotFound desc = could not find container \"7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e\": container with ID starting with 7cf91a2d59027299600cfa587aa66241fe229b28799af7b2caf831aba025f21e not found: ID does not exist"
Oct 30 21:44:02.372174 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.372146    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"]
Oct 30 21:44:02.384804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:02.384766    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-0a4eab/distinct-w5-0a4eab-kserve-67b579955f-nn2z7"]
Oct 30 21:44:04.368869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:04.368836    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" path="/var/lib/kubelet/pods/2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c/volumes"
Oct 30 21:44:07.169819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:07.169790    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:44:22.336320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:22.336285    5786 generic.go:356] "Generic (PLEG): container finished" podID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerID="53373e78d3d418c17cbf22bade2d68fa8be6a6610ec5ae3488f1bb2b31175fbc" exitCode=0
Oct 30 21:44:22.336757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:22.336350    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" event={"ID":"0ddcb89a-a9a5-4798-b1c0-24d7da39df87","Type":"ContainerDied","Data":"53373e78d3d418c17cbf22bade2d68fa8be6a6610ec5ae3488f1bb2b31175fbc"}
Oct 30 21:44:22.336757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:22.336734    5786 scope.go:122] "RemoveContainer" containerID="53373e78d3d418c17cbf22bade2d68fa8be6a6610ec5ae3488f1bb2b31175fbc"
Oct 30 21:44:22.338336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:22.338307    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:44:22.338442 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:22.338376    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:44:22.338442 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:22.338407    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:44:22.356678 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:44:22.356649    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:44:23.346404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:23.346370    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" event={"ID":"0ddcb89a-a9a5-4798-b1c0-24d7da39df87","Type":"ContainerStarted","Data":"6eea36ee4004a7a717e1fec87c3c102b42d3ddafeba08cc3287aaa7482aadf6c"}
Oct 30 21:44:23.443073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:23.443039    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:44:23.443936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:23.443915    5786 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:44:23.443997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:23.443958    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:44:24.442974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:24.442940    5786 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:44:24.443477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:24.442986    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:44:25.443393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:25.443359    5786 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:44:25.443788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:25.443407    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:44:26.166326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:26.166292    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:44:26.443543 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:26.443465    5786 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:44:26.443989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:26.443533    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:44:26.637023 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:26.636994    5786 scope.go:122] "RemoveContainer" containerID="74b031ca7917bbdccaf003074d0205bfdf3f8ec0e2a47e512b0e15e1c6665e1f"
Oct 30 21:44:26.647044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:26.647029    5786 scope.go:122] "RemoveContainer" containerID="5a46a31a03769f733cd2ffc618af355c406e53ea2c1d67b571e08f23f93c76ea"
Oct 30 21:44:27.442957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:27.442925    5786 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:44:27.443125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:27.442983    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:44:28.447729 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:28.447701    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:44:28.448956 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:28.448935    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:44:33.063810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:33.063773    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"]
Oct 30 21:44:33.102692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:33.102663    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-520ea9d9-tsk26"]
Oct 30 21:44:34.361247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:34.361197    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54108272-5457-4321-8ecb-2a34adad8a16" path="/var/lib/kubelet/pods/54108272-5457-4321-8ecb-2a34adad8a16/volumes"
Oct 30 21:44:40.032735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:40.032702    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"]
Oct 30 21:44:40.100381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:40.100341    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-847c62-rwcrs"]
Oct 30 21:44:40.400330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:40.400247    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7128c5ce-3dcc-4743-ad9b-f63b53a2b523" path="/var/lib/kubelet/pods/7128c5ce-3dcc-4743-ad9b-f63b53a2b523/volumes"
Oct 30 21:44:48.182318 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:48.182277    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"]
Oct 30 21:44:48.182815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:48.182561    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy" containerID="cri-o://ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f" gracePeriod=30
Oct 30 21:44:49.371568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.365285    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"]
Oct 30 21:44:49.371568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.366523    5786 cpu_manager.go:414] "containerMap: removing container" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main"
Oct 30 21:44:49.371568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.366542    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main"
Oct 30 21:44:49.371568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.366770    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="2eae98b6-4fb9-4e5f-9fe8-dfe21dd1363c" containerName="main"
Oct 30 21:44:49.372977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.372951    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.377139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.377114    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"]
Oct 30 21:44:49.402706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.402667    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"]
Oct 30 21:44:49.407130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.407107    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.424870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.424843    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:44:49.425044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.425010    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:44:49.442148 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.442125    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"]
Oct 30 21:44:49.559362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.559332    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6dp4\" (UniqueName: \"kubernetes.io/projected/0e333545-99c8-4bc6-a8e8-4be47f9ed805-kube-api-access-q6dp4\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.559532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.559388    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.559532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.559471    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgd4\" (UniqueName: \"kubernetes.io/projected/b814d735-eaf1-4031-b2de-eaad057d8d8d-kube-api-access-slgd4\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.559532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.559509    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.559917 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.559894    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.560041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.560022    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-xks-ca\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.661533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.661454    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.661533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.661527    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-xks-ca\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.661756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.661576    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6dp4\" (UniqueName: \"kubernetes.io/projected/0e333545-99c8-4bc6-a8e8-4be47f9ed805-kube-api-access-q6dp4\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.661756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.661626    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.661756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.661658    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgd4\" (UniqueName: \"kubernetes.io/projected/b814d735-eaf1-4031-b2de-eaad057d8d8d-kube-api-access-slgd4\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.661920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.661786    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.662396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.662372    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.662503 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.662448    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.663875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.663855    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.663964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.663890    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-xks-ca\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.717915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.717890    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6dp4\" (UniqueName: \"kubernetes.io/projected/0e333545-99c8-4bc6-a8e8-4be47f9ed805-kube-api-access-q6dp4\") pod \"maas-api-revoke-sub-14c2cd68-vv5qx\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:49.718083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.718067    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgd4\" (UniqueName: \"kubernetes.io/projected/b814d735-eaf1-4031-b2de-eaad057d8d8d-kube-api-access-slgd4\") pod \"maas-api-revoke-sub-b3ed0a6b-pqxls\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.723937 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.723922    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:49.997382 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:49.997354    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:50.101685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:50.101649    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"]
Oct 30 21:44:50.242429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:50.242392    5786 patch_prober.go:29] interesting pod/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:44:50.242559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:50.242463    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:44:50.377600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:50.377575    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"]
Oct 30 21:44:50.378433 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:44:50.378407    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e333545_99c8_4bc6_a8e8_4be47f9ed805.slice/crio-69d5824a794caa3268ce7fc32bbcc332571a3e211bcb9451ae2aa54f90d534e8 WatchSource:0}: Error finding container 69d5824a794caa3268ce7fc32bbcc332571a3e211bcb9451ae2aa54f90d534e8: Status 404 returned error can't find the container with id 69d5824a794caa3268ce7fc32bbcc332571a3e211bcb9451ae2aa54f90d534e8
Oct 30 21:44:50.603547 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:50.603472    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls" event={"ID":"b814d735-eaf1-4031-b2de-eaad057d8d8d","Type":"ContainerStarted","Data":"95f753cfe39ee732efed5d0689fb53843d3cad170b8d3dc1d7701fc685b8d701"}
Oct 30 21:44:50.603771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:50.603744    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls" event={"ID":"b814d735-eaf1-4031-b2de-eaad057d8d8d","Type":"ContainerStarted","Data":"14061a23192c04f8b6eb1bfc6d66187949ae4ceabf06e89381f7d1f1772a8872"}
Oct 30 21:44:50.605245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:50.605217    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx" event={"ID":"0e333545-99c8-4bc6-a8e8-4be47f9ed805","Type":"ContainerStarted","Data":"69d5824a794caa3268ce7fc32bbcc332571a3e211bcb9451ae2aa54f90d534e8"}
Oct 30 21:44:50.722234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:50.722181    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls" podStartSLOduration=1.722168232 podStartE2EDuration="1.722168232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:50.718591736 +0000 UTC m=+1946.589710854" watchObservedRunningTime="2026-10-30 21:44:50.722168232 +0000 UTC m=+1946.593287372"
Oct 30 21:44:51.614179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:51.614146    5786 generic.go:356] "Generic (PLEG): container finished" podID="b814d735-eaf1-4031-b2de-eaad057d8d8d" containerID="95f753cfe39ee732efed5d0689fb53843d3cad170b8d3dc1d7701fc685b8d701" exitCode=0
Oct 30 21:44:51.614619 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:51.614249    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls" event={"ID":"b814d735-eaf1-4031-b2de-eaad057d8d8d","Type":"ContainerDied","Data":"95f753cfe39ee732efed5d0689fb53843d3cad170b8d3dc1d7701fc685b8d701"}
Oct 30 21:44:51.616105 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:51.616077    5786 generic.go:356] "Generic (PLEG): container finished" podID="0e333545-99c8-4bc6-a8e8-4be47f9ed805" containerID="83bee20273b25c6023d4a81f59cb4d7c7e2e824bf0a6bac381876ccfdac8ea9a" exitCode=0
Oct 30 21:44:51.616235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:51.616127    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx" event={"ID":"0e333545-99c8-4bc6-a8e8-4be47f9ed805","Type":"ContainerDied","Data":"83bee20273b25c6023d4a81f59cb4d7c7e2e824bf0a6bac381876ccfdac8ea9a"}
Oct 30 21:44:52.356026 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:44:52.355994    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:44:52.805055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.805032    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:52.808702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.808682    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:52.829256 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.829235    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-xks-ca\") pod \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") "
Oct 30 21:44:52.829391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.829297    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-openshift-ca\") pod \"b814d735-eaf1-4031-b2de-eaad057d8d8d\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") "
Oct 30 21:44:52.829459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.829408    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e333545-99c8-4bc6-a8e8-4be47f9ed805-kube-api-access-q6dp4\" (UniqueName: \"kubernetes.io/projected/0e333545-99c8-4bc6-a8e8-4be47f9ed805-kube-api-access-q6dp4\") pod \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") "
Oct 30 21:44:52.829459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.829443    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b814d735-eaf1-4031-b2de-eaad057d8d8d-kube-api-access-slgd4\" (UniqueName: \"kubernetes.io/projected/b814d735-eaf1-4031-b2de-eaad057d8d8d-kube-api-access-slgd4\") pod \"b814d735-eaf1-4031-b2de-eaad057d8d8d\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") "
Oct 30 21:44:52.829571 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.829556    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-xks-ca\") pod \"b814d735-eaf1-4031-b2de-eaad057d8d8d\" (UID: \"b814d735-eaf1-4031-b2de-eaad057d8d8d\") "
Oct 30 21:44:52.829642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.829592    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-openshift-ca\") pod \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\" (UID: \"0e333545-99c8-4bc6-a8e8-4be47f9ed805\") "
Oct 30 21:44:52.829696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.829679    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-openshift-ca" pod "b814d735-eaf1-4031-b2de-eaad057d8d8d" (UID: "b814d735-eaf1-4031-b2de-eaad057d8d8d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:52.830028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.829964    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-openshift-ca" pod "0e333545-99c8-4bc6-a8e8-4be47f9ed805" (UID: "0e333545-99c8-4bc6-a8e8-4be47f9ed805"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:52.830347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.830329    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:52.830417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.830351    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:52.831594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.831569    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e333545-99c8-4bc6-a8e8-4be47f9ed805-kube-api-access-q6dp4" pod "0e333545-99c8-4bc6-a8e8-4be47f9ed805" (UID: "0e333545-99c8-4bc6-a8e8-4be47f9ed805"). InnerVolumeSpecName "kube-api-access-q6dp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:52.831697 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.831651    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b814d735-eaf1-4031-b2de-eaad057d8d8d-kube-api-access-slgd4" pod "b814d735-eaf1-4031-b2de-eaad057d8d8d" (UID: "b814d735-eaf1-4031-b2de-eaad057d8d8d"). InnerVolumeSpecName "kube-api-access-slgd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:52.831928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.831916    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-xks-ca" pod "0e333545-99c8-4bc6-a8e8-4be47f9ed805" (UID: "0e333545-99c8-4bc6-a8e8-4be47f9ed805"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:52.831991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.831972    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-xks-ca" pod "b814d735-eaf1-4031-b2de-eaad057d8d8d" (UID: "b814d735-eaf1-4031-b2de-eaad057d8d8d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:52.932187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.932161    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e333545-99c8-4bc6-a8e8-4be47f9ed805-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:52.932187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.932187    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6dp4\" (UniqueName: \"kubernetes.io/projected/0e333545-99c8-4bc6-a8e8-4be47f9ed805-kube-api-access-q6dp4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:52.932434 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.932198    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slgd4\" (UniqueName: \"kubernetes.io/projected/b814d735-eaf1-4031-b2de-eaad057d8d8d-kube-api-access-slgd4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:52.932434 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:52.932229    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b814d735-eaf1-4031-b2de-eaad057d8d8d-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.426568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.426543    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:44:53.538967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.538909    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-certs\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.538967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.538944    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-envoy\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.539185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539057    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-credential-socket\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.539185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539095    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-data\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.539185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539125    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-podinfo\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.539185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539165    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istiod-ca-cert\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.539421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539187    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-token\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-token\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.539421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539193    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-certs" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:53.539421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539284    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-socket\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.539421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539309    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-kube-api-access-xwzvl\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-kube-api-access-xwzvl\") pod \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\" (UID: \"2a36ca42-23dc-46ef-a105-cf2d9ee227f0\") "
Oct 30 21:44:53.539639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539424    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-credential-socket" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:53.539639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539432    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-data" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:53.539735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539653    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-socket" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:53.539735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539681    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istiod-ca-cert" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:53.539870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539854    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.539919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539874    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.539919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539884    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.539919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539893    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.539919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.539901    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.541869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.541843    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-envoy" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:53.541869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.541857    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-podinfo" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:44:53.542096 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.542079    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-kube-api-access-xwzvl" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "kube-api-access-xwzvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:53.542146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.542105    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-token" pod "2a36ca42-23dc-46ef-a105-cf2d9ee227f0" (UID: "2a36ca42-23dc-46ef-a105-cf2d9ee227f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:53.636622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.636593    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"
Oct 30 21:44:53.636795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.636614    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls" event={"ID":"b814d735-eaf1-4031-b2de-eaad057d8d8d","Type":"ContainerDied","Data":"14061a23192c04f8b6eb1bfc6d66187949ae4ceabf06e89381f7d1f1772a8872"}
Oct 30 21:44:53.636795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.636672    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14061a23192c04f8b6eb1bfc6d66187949ae4ceabf06e89381f7d1f1772a8872"
Oct 30 21:44:53.638376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.638349    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"
Oct 30 21:44:53.638520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.638388    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx" event={"ID":"0e333545-99c8-4bc6-a8e8-4be47f9ed805","Type":"ContainerDied","Data":"69d5824a794caa3268ce7fc32bbcc332571a3e211bcb9451ae2aa54f90d534e8"}
Oct 30 21:44:53.638520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.638436    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69d5824a794caa3268ce7fc32bbcc332571a3e211bcb9451ae2aa54f90d534e8"
Oct 30 21:44:53.642389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.641851    5786 generic.go:356] "Generic (PLEG): container finished" podID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerID="ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f" exitCode=0
Oct 30 21:44:53.642389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.641969    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" event={"ID":"2a36ca42-23dc-46ef-a105-cf2d9ee227f0","Type":"ContainerDied","Data":"ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f"}
Oct 30 21:44:53.642389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.642008    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6" event={"ID":"2a36ca42-23dc-46ef-a105-cf2d9ee227f0","Type":"ContainerDied","Data":"db8a955e2bb64189e8c2d59b24037972c6c8d81c905d03f6007fa3ce2f8b488e"}
Oct 30 21:44:53.642389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.642038    5786 scope.go:122] "RemoveContainer" containerID="ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f"
Oct 30 21:44:53.642389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.642183    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"
Oct 30 21:44:53.642389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.642349    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.642857 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.642452    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.642857 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.642808    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwzvl\" (UniqueName: \"kubernetes.io/projected/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-kube-api-access-xwzvl\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.642857 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.642828    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a36ca42-23dc-46ef-a105-cf2d9ee227f0-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:53.658461 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.658441    5786 scope.go:122] "RemoveContainer" containerID="fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e"
Oct 30 21:44:53.667888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.667861    5786 scope.go:122] "RemoveContainer" containerID="ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f"
Oct 30 21:44:53.668143 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:44:53.668124    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f\": container with ID starting with ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f not found: ID does not exist" containerID="ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f"
Oct 30 21:44:53.668191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.668150    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f"} err="failed to get container status \"ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f\": rpc error: code = NotFound desc = could not find container \"ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f\": container with ID starting with ceb28bfeedd327aadcb4a2c1e9d28e90926681bcb79677f8212983a975e4948f not found: ID does not exist"
Oct 30 21:44:53.668191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.668166    5786 scope.go:122] "RemoveContainer" containerID="fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e"
Oct 30 21:44:53.668414 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:44:53.668395    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e\": container with ID starting with fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e not found: ID does not exist" containerID="fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e"
Oct 30 21:44:53.668458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.668421    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e"} err="failed to get container status \"fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e\": rpc error: code = NotFound desc = could not find container \"fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e\": container with ID starting with fc754e1e7f63ed85676a470dc9d7ec26673208018f5e3ddebe6068bd36250f1e not found: ID does not exist"
Oct 30 21:44:53.808409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.808351    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"]
Oct 30 21:44:53.813706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:53.813680    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-847c62-openshift-default-6bc5bf44df-wqnv6"]
Oct 30 21:44:54.361841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:44:54.361811    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" path="/var/lib/kubelet/pods/2a36ca42-23dc-46ef-a105-cf2d9ee227f0/volumes"
Oct 30 21:45:00.181232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.180265    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"]
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181372    5786 cpu_manager.go:414] "containerMap: removing container" podUID="0e333545-99c8-4bc6-a8e8-4be47f9ed805" containerName="revoke-keys"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181395    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e333545-99c8-4bc6-a8e8-4be47f9ed805" containerName="revoke-keys"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181413    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b814d735-eaf1-4031-b2de-eaad057d8d8d" containerName="revoke-keys"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181421    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b814d735-eaf1-4031-b2de-eaad057d8d8d" containerName="revoke-keys"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181441    5786 cpu_manager.go:414] "containerMap: removing container" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181449    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181483    5786 cpu_manager.go:414] "containerMap: removing container" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181490    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181681    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e333545-99c8-4bc6-a8e8-4be47f9ed805" containerName="revoke-keys"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181693    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b814d735-eaf1-4031-b2de-eaad057d8d8d" containerName="revoke-keys"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181707    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy"
Oct 30 21:45:00.181919 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.181720    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a36ca42-23dc-46ef-a105-cf2d9ee227f0" containerName="istio-proxy"
Oct 30 21:45:00.198629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.198601    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:00.199006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.198855    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"]
Oct 30 21:45:00.233466 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.233437    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"]
Oct 30 21:45:00.237469 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.237444    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:00.254232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.242520    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qvmh\" (UniqueName: \"kubernetes.io/projected/40f37fc3-dedd-467b-8aa3-b455273e8cf3-kube-api-access-2qvmh\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr\" (UID: \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:00.254232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.242865    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f37fc3-dedd-467b-8aa3-b455273e8cf3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr\" (UID: \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:00.258443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.258401    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"]
Oct 30 21:45:00.262942 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.259900    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:00.267118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.267085    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"]
Oct 30 21:45:00.267263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.267199    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:00.275806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.274337    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"]
Oct 30 21:45:00.275806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.274446    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:00.279893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.279875    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"]
Oct 30 21:45:00.280068 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.279914    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"]
Oct 30 21:45:00.280470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.280115    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:00.290670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.290645    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"]
Oct 30 21:45:00.340260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.330980    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:45:00.340260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.334280    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"]
Oct 30 21:45:00.340260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.336676    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"]
Oct 30 21:45:00.344190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344162    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4dqr\" (UniqueName: \"kubernetes.io/projected/7b6a7006-c5dc-43d4-af26-37bb73e1c593-kube-api-access-r4dqr\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865\" (UID: \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:00.344335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344233    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5695454a-c88c-48cd-aba4-207d3bbe97a3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889945-2d9nv\" (UID: \"5695454a-c88c-48cd-aba4-207d3bbe97a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:00.344335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344285    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d325fad6-efa5-4bd8-a836-b664ec176dab-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f\" (UID: \"d325fad6-efa5-4bd8-a836-b664ec176dab\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:00.344416 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344397    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbpmh\" (UniqueName: \"kubernetes.io/projected/5695454a-c88c-48cd-aba4-207d3bbe97a3-kube-api-access-mbpmh\") pod \"maas-api-key-cleanup-29889945-2d9nv\" (UID: \"5695454a-c88c-48cd-aba4-207d3bbe97a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:00.344458 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344436    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lrh\" (UniqueName: \"kubernetes.io/projected/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-kube-api-access-h2lrh\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6\" (UID: \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:00.344502 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344491    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71da857c-d45b-4779-a63a-bd5cb8e9ba48-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx\" (UID: \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:00.344537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344517    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmg8m\" (UniqueName: \"kubernetes.io/projected/71da857c-d45b-4779-a63a-bd5cb8e9ba48-kube-api-access-zmg8m\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx\" (UID: \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:00.345083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344585    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f37fc3-dedd-467b-8aa3-b455273e8cf3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr\" (UID: \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:00.345083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344762    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvmh\" (UniqueName: \"kubernetes.io/projected/40f37fc3-dedd-467b-8aa3-b455273e8cf3-kube-api-access-2qvmh\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr\" (UID: \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:00.345083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344797    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p62nl\" (UniqueName: \"kubernetes.io/projected/d325fad6-efa5-4bd8-a836-b664ec176dab-kube-api-access-p62nl\") pod \"maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f\" (UID: \"d325fad6-efa5-4bd8-a836-b664ec176dab\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:00.345083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344833    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6a7006-c5dc-43d4-af26-37bb73e1c593-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865\" (UID: \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:00.345083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.344885    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6\" (UID: \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:00.345364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.345341    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f37fc3-dedd-467b-8aa3-b455273e8cf3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr\" (UID: \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:00.370851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.370826    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"]
Oct 30 21:45:00.402939 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.402913    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvmh\" (UniqueName: \"kubernetes.io/projected/40f37fc3-dedd-467b-8aa3-b455273e8cf3-kube-api-access-2qvmh\") pod \"maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr\" (UID: \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:00.446822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.446765    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p62nl\" (UniqueName: \"kubernetes.io/projected/d325fad6-efa5-4bd8-a836-b664ec176dab-kube-api-access-p62nl\") pod \"maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f\" (UID: \"d325fad6-efa5-4bd8-a836-b664ec176dab\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:00.446822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.446799    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6a7006-c5dc-43d4-af26-37bb73e1c593-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865\" (UID: \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:00.447013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.446831    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6\" (UID: \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:00.447013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.446885    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4dqr\" (UniqueName: \"kubernetes.io/projected/7b6a7006-c5dc-43d4-af26-37bb73e1c593-kube-api-access-r4dqr\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865\" (UID: \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:00.447013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.446910    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5695454a-c88c-48cd-aba4-207d3bbe97a3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889945-2d9nv\" (UID: \"5695454a-c88c-48cd-aba4-207d3bbe97a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:00.447013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.446945    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d325fad6-efa5-4bd8-a836-b664ec176dab-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f\" (UID: \"d325fad6-efa5-4bd8-a836-b664ec176dab\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:00.447265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447039    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbpmh\" (UniqueName: \"kubernetes.io/projected/5695454a-c88c-48cd-aba4-207d3bbe97a3-kube-api-access-mbpmh\") pod \"maas-api-key-cleanup-29889945-2d9nv\" (UID: \"5695454a-c88c-48cd-aba4-207d3bbe97a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:00.447265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447122    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lrh\" (UniqueName: \"kubernetes.io/projected/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-kube-api-access-h2lrh\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6\" (UID: \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:00.447265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447195    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71da857c-d45b-4779-a63a-bd5cb8e9ba48-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx\" (UID: \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:00.447265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447243    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmg8m\" (UniqueName: \"kubernetes.io/projected/71da857c-d45b-4779-a63a-bd5cb8e9ba48-kube-api-access-zmg8m\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx\" (UID: \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:00.447626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447605    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6a7006-c5dc-43d4-af26-37bb73e1c593-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865\" (UID: \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:00.447702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447639    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6\" (UID: \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:00.447740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447713    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5695454a-c88c-48cd-aba4-207d3bbe97a3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889945-2d9nv\" (UID: \"5695454a-c88c-48cd-aba4-207d3bbe97a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:00.447785 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447768    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d325fad6-efa5-4bd8-a836-b664ec176dab-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f\" (UID: \"d325fad6-efa5-4bd8-a836-b664ec176dab\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:00.447965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.447948    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71da857c-d45b-4779-a63a-bd5cb8e9ba48-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx\" (UID: \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:00.503920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.503892    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmg8m\" (UniqueName: \"kubernetes.io/projected/71da857c-d45b-4779-a63a-bd5cb8e9ba48-kube-api-access-zmg8m\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx\" (UID: \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:00.504025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.503892    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p62nl\" (UniqueName: \"kubernetes.io/projected/d325fad6-efa5-4bd8-a836-b664ec176dab-kube-api-access-p62nl\") pod \"maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f\" (UID: \"d325fad6-efa5-4bd8-a836-b664ec176dab\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:00.504136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.504120    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lrh\" (UniqueName: \"kubernetes.io/projected/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-kube-api-access-h2lrh\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6\" (UID: \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:00.504247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.504228    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbpmh\" (UniqueName: \"kubernetes.io/projected/5695454a-c88c-48cd-aba4-207d3bbe97a3-kube-api-access-mbpmh\") pod \"maas-api-key-cleanup-29889945-2d9nv\" (UID: \"5695454a-c88c-48cd-aba4-207d3bbe97a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:00.504362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.504347    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4dqr\" (UniqueName: \"kubernetes.io/projected/7b6a7006-c5dc-43d4-af26-37bb73e1c593-kube-api-access-r4dqr\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865\" (UID: \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:00.566196 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.566177    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:00.582460 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.582433    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:00.591792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.591771    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:00.599955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.599931    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:00.614541 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.614516    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:00.616620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.616512    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:00.959158 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:00.959107    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"]
Oct 30 21:45:00.960339 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:45:00.960310    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f37fc3_dedd_467b_8aa3_b455273e8cf3.slice/crio-9880b00ceff937438e20121b87e5a113744a56d0bfbcfddedd727bc36fda9ef7 WatchSource:0}: Error finding container 9880b00ceff937438e20121b87e5a113744a56d0bfbcfddedd727bc36fda9ef7: Status 404 returned error can't find the container with id 9880b00ceff937438e20121b87e5a113744a56d0bfbcfddedd727bc36fda9ef7
Oct 30 21:45:01.009243 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.009199    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"]
Oct 30 21:45:01.012326 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:45:01.012303    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf6a8a8_b9f1_4d7b_9de9_29f3fc99d7ac.slice/crio-ed37e46107d02d4da5c9bdb28f42f6ec4b3fe09b5ce660f6d75e3c44450d6e54 WatchSource:0}: Error finding container ed37e46107d02d4da5c9bdb28f42f6ec4b3fe09b5ce660f6d75e3c44450d6e54: Status 404 returned error can't find the container with id ed37e46107d02d4da5c9bdb28f42f6ec4b3fe09b5ce660f6d75e3c44450d6e54
Oct 30 21:45:01.044034 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:45:01.044006    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd325fad6_efa5_4bd8_a836_b664ec176dab.slice/crio-4902bdc13946d3b1cd7b37011c2213d5f37572691c16f51209bb5bf59c4e8c66 WatchSource:0}: Error finding container 4902bdc13946d3b1cd7b37011c2213d5f37572691c16f51209bb5bf59c4e8c66: Status 404 returned error can't find the container with id 4902bdc13946d3b1cd7b37011c2213d5f37572691c16f51209bb5bf59c4e8c66
Oct 30 21:45:01.044140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.044111    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"]
Oct 30 21:45:01.060924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.060841    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"]
Oct 30 21:45:01.068945 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.068807    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"]
Oct 30 21:45:01.115854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.115816    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"]
Oct 30 21:45:01.720766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.720735    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" event={"ID":"5695454a-c88c-48cd-aba4-207d3bbe97a3","Type":"ContainerStarted","Data":"834980b4f8355fd36f42fec4e10b0914b69d9e98e929c7733bf85cefff2bc605"}
Oct 30 21:45:01.721133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.720789    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" event={"ID":"5695454a-c88c-48cd-aba4-207d3bbe97a3","Type":"ContainerStarted","Data":"c005c1bdd79c65e44664ca51dfe095e71345319fec11e4ab35a7d2eadbac4321"}
Oct 30 21:45:01.722421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.722396    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" event={"ID":"d325fad6-efa5-4bd8-a836-b664ec176dab","Type":"ContainerStarted","Data":"6504dce481e03cad51877527fee291e43ac2cf59853fc363a46056f219f0da7a"}
Oct 30 21:45:01.722522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.722443    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" event={"ID":"d325fad6-efa5-4bd8-a836-b664ec176dab","Type":"ContainerStarted","Data":"4902bdc13946d3b1cd7b37011c2213d5f37572691c16f51209bb5bf59c4e8c66"}
Oct 30 21:45:01.723918 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.723897    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" event={"ID":"40f37fc3-dedd-467b-8aa3-b455273e8cf3","Type":"ContainerStarted","Data":"bc50c77ae31889d27966b69240cb467c2a24f9472dbddcd4566bf8ceea2cd76a"}
Oct 30 21:45:01.723998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.723938    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" event={"ID":"40f37fc3-dedd-467b-8aa3-b455273e8cf3","Type":"ContainerStarted","Data":"9880b00ceff937438e20121b87e5a113744a56d0bfbcfddedd727bc36fda9ef7"}
Oct 30 21:45:01.725415 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.725397    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" event={"ID":"71da857c-d45b-4779-a63a-bd5cb8e9ba48","Type":"ContainerStarted","Data":"e11d82e19e692a30c446b58f1941d7078da3608ff1fb46cc667467b1a05dc1e8"}
Oct 30 21:45:01.725506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.725437    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" event={"ID":"71da857c-d45b-4779-a63a-bd5cb8e9ba48","Type":"ContainerStarted","Data":"f4dcf56ab3af0bda3a53b3ca890d36351f1ce17784b95d3219c2f967dd987674"}
Oct 30 21:45:01.726844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.726824    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" event={"ID":"7b6a7006-c5dc-43d4-af26-37bb73e1c593","Type":"ContainerStarted","Data":"2cfc0bb02d8fa8f7b32b66f2de43686f4669f0483e21cdd26d26feb41b38a096"}
Oct 30 21:45:01.726952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.726859    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" event={"ID":"7b6a7006-c5dc-43d4-af26-37bb73e1c593","Type":"ContainerStarted","Data":"8b0aa785de3e525be29ccfbd3d235823c85219a3496eca0d745d2ab65bd5e1ad"}
Oct 30 21:45:01.728252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.728232    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" event={"ID":"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac","Type":"ContainerStarted","Data":"c9bd435f5f88eaf69ab801f911874c1f5e32f63f0f658b17bef2b4716bb77c0e"}
Oct 30 21:45:01.728346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.728268    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" event={"ID":"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac","Type":"ContainerStarted","Data":"ed37e46107d02d4da5c9bdb28f42f6ec4b3fe09b5ce660f6d75e3c44450d6e54"}
Oct 30 21:45:01.835663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.835561    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" podStartSLOduration=1.835549096 podStartE2EDuration="1.835549096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:01.83350431 +0000 UTC m=+1957.704623430" watchObservedRunningTime="2026-10-30 21:45:01.835549096 +0000 UTC m=+1957.706668215"
Oct 30 21:45:01.947545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:01.947497    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" podStartSLOduration=1.9474847130000001 podStartE2EDuration="1.947484713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:01.94577906 +0000 UTC m=+1957.816898181" watchObservedRunningTime="2026-10-30 21:45:01.947484713 +0000 UTC m=+1957.818603831"
Oct 30 21:45:02.190965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.190889    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" podStartSLOduration=2.190870886 podStartE2EDuration="2.190870886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.189930147 +0000 UTC m=+1958.061049267" watchObservedRunningTime="2026-10-30 21:45:02.190870886 +0000 UTC m=+1958.061990004"
Oct 30 21:45:02.194515 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.194470    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" podStartSLOduration=2.194457534 podStartE2EDuration="2.194457534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.073356077 +0000 UTC m=+1957.944475198" watchObservedRunningTime="2026-10-30 21:45:02.194457534 +0000 UTC m=+1958.065576692"
Oct 30 21:45:02.313293 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.309114    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" podStartSLOduration=2.30909722 podStartE2EDuration="2.30909722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.308897168 +0000 UTC m=+1958.180016286" watchObservedRunningTime="2026-10-30 21:45:02.30909722 +0000 UTC m=+1958.180216341"
Oct 30 21:45:02.427116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.427072    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" podStartSLOduration=2.427059933 podStartE2EDuration="2.427059933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.42585693 +0000 UTC m=+1958.296976049" watchObservedRunningTime="2026-10-30 21:45:02.427059933 +0000 UTC m=+1958.298179086"
Oct 30 21:45:02.737501 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.737409    5786 generic.go:356] "Generic (PLEG): container finished" podID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerID="bc50c77ae31889d27966b69240cb467c2a24f9472dbddcd4566bf8ceea2cd76a" exitCode=60
Oct 30 21:45:02.737933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.737492    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" event={"ID":"40f37fc3-dedd-467b-8aa3-b455273e8cf3","Type":"ContainerDied","Data":"bc50c77ae31889d27966b69240cb467c2a24f9472dbddcd4566bf8ceea2cd76a"}
Oct 30 21:45:02.737933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.737730    5786 scope.go:122] "RemoveContainer" containerID="bc50c77ae31889d27966b69240cb467c2a24f9472dbddcd4566bf8ceea2cd76a"
Oct 30 21:45:02.739973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.739952    5786 generic.go:356] "Generic (PLEG): container finished" podID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerID="e11d82e19e692a30c446b58f1941d7078da3608ff1fb46cc667467b1a05dc1e8" exitCode=60
Oct 30 21:45:02.740084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.740030    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" event={"ID":"71da857c-d45b-4779-a63a-bd5cb8e9ba48","Type":"ContainerDied","Data":"e11d82e19e692a30c446b58f1941d7078da3608ff1fb46cc667467b1a05dc1e8"}
Oct 30 21:45:02.740320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.740305    5786 scope.go:122] "RemoveContainer" containerID="e11d82e19e692a30c446b58f1941d7078da3608ff1fb46cc667467b1a05dc1e8"
Oct 30 21:45:02.741926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.741904    5786 generic.go:356] "Generic (PLEG): container finished" podID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerID="2cfc0bb02d8fa8f7b32b66f2de43686f4669f0483e21cdd26d26feb41b38a096" exitCode=60
Oct 30 21:45:02.742037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.742006    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" event={"ID":"7b6a7006-c5dc-43d4-af26-37bb73e1c593","Type":"ContainerDied","Data":"2cfc0bb02d8fa8f7b32b66f2de43686f4669f0483e21cdd26d26feb41b38a096"}
Oct 30 21:45:02.742225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.742194    5786 scope.go:122] "RemoveContainer" containerID="2cfc0bb02d8fa8f7b32b66f2de43686f4669f0483e21cdd26d26feb41b38a096"
Oct 30 21:45:02.743776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.743758    5786 generic.go:356] "Generic (PLEG): container finished" podID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerID="c9bd435f5f88eaf69ab801f911874c1f5e32f63f0f658b17bef2b4716bb77c0e" exitCode=60
Oct 30 21:45:02.743892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.743834    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" event={"ID":"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac","Type":"ContainerDied","Data":"c9bd435f5f88eaf69ab801f911874c1f5e32f63f0f658b17bef2b4716bb77c0e"}
Oct 30 21:45:02.744064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.744047    5786 scope.go:122] "RemoveContainer" containerID="c9bd435f5f88eaf69ab801f911874c1f5e32f63f0f658b17bef2b4716bb77c0e"
Oct 30 21:45:02.745544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.745523    5786 generic.go:356] "Generic (PLEG): container finished" podID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerID="834980b4f8355fd36f42fec4e10b0914b69d9e98e929c7733bf85cefff2bc605" exitCode=60
Oct 30 21:45:02.745661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.745549    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" event={"ID":"5695454a-c88c-48cd-aba4-207d3bbe97a3","Type":"ContainerDied","Data":"834980b4f8355fd36f42fec4e10b0914b69d9e98e929c7733bf85cefff2bc605"}
Oct 30 21:45:02.745792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.745778    5786 scope.go:122] "RemoveContainer" containerID="834980b4f8355fd36f42fec4e10b0914b69d9e98e929c7733bf85cefff2bc605"
Oct 30 21:45:02.747536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.747515    5786 generic.go:356] "Generic (PLEG): container finished" podID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerID="6504dce481e03cad51877527fee291e43ac2cf59853fc363a46056f219f0da7a" exitCode=60
Oct 30 21:45:02.747636 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.747570    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" event={"ID":"d325fad6-efa5-4bd8-a836-b664ec176dab","Type":"ContainerDied","Data":"6504dce481e03cad51877527fee291e43ac2cf59853fc363a46056f219f0da7a"}
Oct 30 21:45:02.747803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:02.747788    5786 scope.go:122] "RemoveContainer" containerID="6504dce481e03cad51877527fee291e43ac2cf59853fc363a46056f219f0da7a"
Oct 30 21:45:03.783938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:03.783900    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" event={"ID":"5695454a-c88c-48cd-aba4-207d3bbe97a3","Type":"ContainerStarted","Data":"c7a9caf5161ca0da36d2ffe4e7ef32363487499e1cf88cc3da86ff37e515b0ce"}
Oct 30 21:45:03.786286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:03.786257    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" event={"ID":"d325fad6-efa5-4bd8-a836-b664ec176dab","Type":"ContainerStarted","Data":"45f82cf8623f683555d9e4a9d97553bd515cbbd98cede8e888870330f6353b3d"}
Oct 30 21:45:03.787910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:03.787886    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" event={"ID":"40f37fc3-dedd-467b-8aa3-b455273e8cf3","Type":"ContainerStarted","Data":"1b3291ce7fcbf459fb1e9de698a144adf21b2d10b272a25339abb043d3da3d56"}
Oct 30 21:45:03.789664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:03.789645    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" event={"ID":"71da857c-d45b-4779-a63a-bd5cb8e9ba48","Type":"ContainerStarted","Data":"567e8756c96a317597cbdd429bb62ab7b6cfda90e7fda588088d08b32f41290d"}
Oct 30 21:45:03.791413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:03.791392    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" event={"ID":"7b6a7006-c5dc-43d4-af26-37bb73e1c593","Type":"ContainerStarted","Data":"3ce16a3de7d267a134b21f2ae6fd5458ee2dff962cf17e20147899e65986787a"}
Oct 30 21:45:03.793172 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:03.793144    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" event={"ID":"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac","Type":"ContainerStarted","Data":"552cfbc27abf79c7d4985c9e74f88f42db8db56e4568132ab8cf806abb1c95f8"}
Oct 30 21:45:04.802738 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.802708    5786 generic.go:356] "Generic (PLEG): container finished" podID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerID="c7a9caf5161ca0da36d2ffe4e7ef32363487499e1cf88cc3da86ff37e515b0ce" exitCode=60
Oct 30 21:45:04.803238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.802782    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" event={"ID":"5695454a-c88c-48cd-aba4-207d3bbe97a3","Type":"ContainerDied","Data":"c7a9caf5161ca0da36d2ffe4e7ef32363487499e1cf88cc3da86ff37e515b0ce"}
Oct 30 21:45:04.803238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.802832    5786 scope.go:122] "RemoveContainer" containerID="834980b4f8355fd36f42fec4e10b0914b69d9e98e929c7733bf85cefff2bc605"
Oct 30 21:45:04.803238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.803136    5786 scope.go:122] "RemoveContainer" containerID="c7a9caf5161ca0da36d2ffe4e7ef32363487499e1cf88cc3da86ff37e515b0ce"
Oct 30 21:45:04.803569 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:04.803540    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889945-2d9nv_odh-ai-gateway-infra(5695454a-c88c-48cd-aba4-207d3bbe97a3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3"
Oct 30 21:45:04.804948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.804927    5786 generic.go:356] "Generic (PLEG): container finished" podID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerID="45f82cf8623f683555d9e4a9d97553bd515cbbd98cede8e888870330f6353b3d" exitCode=60
Oct 30 21:45:04.805044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.805016    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" event={"ID":"d325fad6-efa5-4bd8-a836-b664ec176dab","Type":"ContainerDied","Data":"45f82cf8623f683555d9e4a9d97553bd515cbbd98cede8e888870330f6353b3d"}
Oct 30 21:45:04.805267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.805252    5786 scope.go:122] "RemoveContainer" containerID="45f82cf8623f683555d9e4a9d97553bd515cbbd98cede8e888870330f6353b3d"
Oct 30 21:45:04.805586 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:04.805561    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f_odh-ai-gateway-infra(d325fad6-efa5-4bd8-a836-b664ec176dab)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab"
Oct 30 21:45:04.806886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.806865    5786 generic.go:356] "Generic (PLEG): container finished" podID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerID="1b3291ce7fcbf459fb1e9de698a144adf21b2d10b272a25339abb043d3da3d56" exitCode=60
Oct 30 21:45:04.806972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.806955    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" event={"ID":"40f37fc3-dedd-467b-8aa3-b455273e8cf3","Type":"ContainerDied","Data":"1b3291ce7fcbf459fb1e9de698a144adf21b2d10b272a25339abb043d3da3d56"}
Oct 30 21:45:04.807183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.807162    5786 scope.go:122] "RemoveContainer" containerID="1b3291ce7fcbf459fb1e9de698a144adf21b2d10b272a25339abb043d3da3d56"
Oct 30 21:45:04.807476 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:04.807456    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr_odh-ai-gateway-infra(40f37fc3-dedd-467b-8aa3-b455273e8cf3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3"
Oct 30 21:45:04.809305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.809286    5786 generic.go:356] "Generic (PLEG): container finished" podID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerID="567e8756c96a317597cbdd429bb62ab7b6cfda90e7fda588088d08b32f41290d" exitCode=60
Oct 30 21:45:04.809403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.809307    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" event={"ID":"71da857c-d45b-4779-a63a-bd5cb8e9ba48","Type":"ContainerDied","Data":"567e8756c96a317597cbdd429bb62ab7b6cfda90e7fda588088d08b32f41290d"}
Oct 30 21:45:04.809570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.809556    5786 scope.go:122] "RemoveContainer" containerID="567e8756c96a317597cbdd429bb62ab7b6cfda90e7fda588088d08b32f41290d"
Oct 30 21:45:04.809842 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:04.809823    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx_odh-ai-gateway-infra(71da857c-d45b-4779-a63a-bd5cb8e9ba48)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48"
Oct 30 21:45:04.811303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.811283    5786 generic.go:356] "Generic (PLEG): container finished" podID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerID="3ce16a3de7d267a134b21f2ae6fd5458ee2dff962cf17e20147899e65986787a" exitCode=60
Oct 30 21:45:04.811400 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.811363    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" event={"ID":"7b6a7006-c5dc-43d4-af26-37bb73e1c593","Type":"ContainerDied","Data":"3ce16a3de7d267a134b21f2ae6fd5458ee2dff962cf17e20147899e65986787a"}
Oct 30 21:45:04.811566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.811550    5786 scope.go:122] "RemoveContainer" containerID="3ce16a3de7d267a134b21f2ae6fd5458ee2dff962cf17e20147899e65986787a"
Oct 30 21:45:04.813127 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.813110    5786 generic.go:356] "Generic (PLEG): container finished" podID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerID="552cfbc27abf79c7d4985c9e74f88f42db8db56e4568132ab8cf806abb1c95f8" exitCode=60
Oct 30 21:45:04.813222 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.813158    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" event={"ID":"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac","Type":"ContainerDied","Data":"552cfbc27abf79c7d4985c9e74f88f42db8db56e4568132ab8cf806abb1c95f8"}
Oct 30 21:45:04.813380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.813367    5786 scope.go:122] "RemoveContainer" containerID="552cfbc27abf79c7d4985c9e74f88f42db8db56e4568132ab8cf806abb1c95f8"
Oct 30 21:45:04.813570 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:04.813556    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6_odh-ai-gateway-infra(bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac"
Oct 30 21:45:04.816533 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:04.816483    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865_odh-ai-gateway-infra(7b6a7006-c5dc-43d4-af26-37bb73e1c593)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593"
Oct 30 21:45:04.817584 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.817565    5786 scope.go:122] "RemoveContainer" containerID="6504dce481e03cad51877527fee291e43ac2cf59853fc363a46056f219f0da7a"
Oct 30 21:45:04.830783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.830764    5786 scope.go:122] "RemoveContainer" containerID="bc50c77ae31889d27966b69240cb467c2a24f9472dbddcd4566bf8ceea2cd76a"
Oct 30 21:45:04.846362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.846347    5786 scope.go:122] "RemoveContainer" containerID="e11d82e19e692a30c446b58f1941d7078da3608ff1fb46cc667467b1a05dc1e8"
Oct 30 21:45:04.884250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.884230    5786 scope.go:122] "RemoveContainer" containerID="2cfc0bb02d8fa8f7b32b66f2de43686f4669f0483e21cdd26d26feb41b38a096"
Oct 30 21:45:04.897271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:04.897251    5786 scope.go:122] "RemoveContainer" containerID="c9bd435f5f88eaf69ab801f911874c1f5e32f63f0f658b17bef2b4716bb77c0e"
Oct 30 21:45:05.821834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:05.821800    5786 scope.go:122] "RemoveContainer" containerID="567e8756c96a317597cbdd429bb62ab7b6cfda90e7fda588088d08b32f41290d"
Oct 30 21:45:05.822319 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:05.822089    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx_odh-ai-gateway-infra(71da857c-d45b-4779-a63a-bd5cb8e9ba48)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48"
Oct 30 21:45:05.823581 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:05.823561    5786 scope.go:122] "RemoveContainer" containerID="3ce16a3de7d267a134b21f2ae6fd5458ee2dff962cf17e20147899e65986787a"
Oct 30 21:45:05.823843 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:05.823814    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865_odh-ai-gateway-infra(7b6a7006-c5dc-43d4-af26-37bb73e1c593)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593"
Oct 30 21:45:05.825059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:05.825038    5786 scope.go:122] "RemoveContainer" containerID="552cfbc27abf79c7d4985c9e74f88f42db8db56e4568132ab8cf806abb1c95f8"
Oct 30 21:45:05.825367 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:05.825344    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6_odh-ai-gateway-infra(bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac"
Oct 30 21:45:05.826800 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:05.826780    5786 scope.go:122] "RemoveContainer" containerID="c7a9caf5161ca0da36d2ffe4e7ef32363487499e1cf88cc3da86ff37e515b0ce"
Oct 30 21:45:05.827012 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:05.826993    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889945-2d9nv_odh-ai-gateway-infra(5695454a-c88c-48cd-aba4-207d3bbe97a3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3"
Oct 30 21:45:05.829282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:05.829264    5786 scope.go:122] "RemoveContainer" containerID="45f82cf8623f683555d9e4a9d97553bd515cbbd98cede8e888870330f6353b3d"
Oct 30 21:45:05.829629 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:05.829596    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f_odh-ai-gateway-infra(d325fad6-efa5-4bd8-a836-b664ec176dab)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab"
Oct 30 21:45:05.830940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:05.830920    5786 scope.go:122] "RemoveContainer" containerID="1b3291ce7fcbf459fb1e9de698a144adf21b2d10b272a25339abb043d3da3d56"
Oct 30 21:45:05.831144 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:05.831123    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr_odh-ai-gateway-infra(40f37fc3-dedd-467b-8aa3-b455273e8cf3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3"
Oct 30 21:45:09.863711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:09.863678    5786 generic.go:356] "Generic (PLEG): container finished" podID="aa8f6777-2d13-4940-b11a-4083c144496d" containerID="efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995" exitCode=0
Oct 30 21:45:09.864060 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:09.863735    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" event={"ID":"aa8f6777-2d13-4940-b11a-4083c144496d","Type":"ContainerDied","Data":"efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995"}
Oct 30 21:45:09.864113 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:09.864096    5786 scope.go:122] "RemoveContainer" containerID="efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995"
Oct 30 21:45:09.865738 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:09.865702    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:45:09.865856 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:09.865796    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:45:09.865856 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:09.865848    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:45:10.882626 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:10.882591    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" event={"ID":"aa8f6777-2d13-4940-b11a-4083c144496d","Type":"ContainerStarted","Data":"8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce"}
Oct 30 21:45:11.791097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:11.791069    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:45:11.876380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:11.876344    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:45:11.877598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:11.877573    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:45:11.877708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:11.877648    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 21:45:12.876231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:12.875528    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:45:12.876231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:12.875586    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 21:45:13.876254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:13.874274    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:45:13.876254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:13.874353    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 21:45:14.380426 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:14.380393    5786 scope.go:122] "RemoveContainer" containerID="552cfbc27abf79c7d4985c9e74f88f42db8db56e4568132ab8cf806abb1c95f8"
Oct 30 21:45:14.876577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:14.876540    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:45:14.876937 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:14.876617    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 21:45:14.914138 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:14.914108    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" event={"ID":"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac","Type":"ContainerStarted","Data":"c31a51362018ecb9bb399850628c9c800b6206b41607fd5a87b612b725cde3a2"}
Oct 30 21:45:15.878971 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:15.878898    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:45:15.880321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:15.880299    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:45:15.923035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:15.923011    5786 generic.go:356] "Generic (PLEG): container finished" podID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerID="c31a51362018ecb9bb399850628c9c800b6206b41607fd5a87b612b725cde3a2" exitCode=60
Oct 30 21:45:15.923129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:15.923082    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" event={"ID":"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac","Type":"ContainerDied","Data":"c31a51362018ecb9bb399850628c9c800b6206b41607fd5a87b612b725cde3a2"}
Oct 30 21:45:15.923177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:15.923130    5786 scope.go:122] "RemoveContainer" containerID="552cfbc27abf79c7d4985c9e74f88f42db8db56e4568132ab8cf806abb1c95f8"
Oct 30 21:45:15.923748 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:15.923730    5786 scope.go:122] "RemoveContainer" containerID="c31a51362018ecb9bb399850628c9c800b6206b41607fd5a87b612b725cde3a2"
Oct 30 21:45:15.924317 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:15.924013    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6_odh-ai-gateway-infra(bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac"
Oct 30 21:45:16.045881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:16.045856    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"]
Oct 30 21:45:17.224280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.210944    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:17.306307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.306270    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-maas-api-service-ca\") pod \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\" (UID: \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\") "
Oct 30 21:45:17.307045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.307023    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-kube-api-access-h2lrh\" (UniqueName: \"kubernetes.io/projected/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-kube-api-access-h2lrh\") pod \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\" (UID: \"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac\") "
Oct 30 21:45:17.312672 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.306923    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-maas-api-service-ca" pod "bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" (UID: "bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:17.363709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.361862    5786 scope.go:122] "RemoveContainer" containerID="3ce16a3de7d267a134b21f2ae6fd5458ee2dff962cf17e20147899e65986787a"
Oct 30 21:45:17.363709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.362789    5786 scope.go:122] "RemoveContainer" containerID="45f82cf8623f683555d9e4a9d97553bd515cbbd98cede8e888870330f6353b3d"
Oct 30 21:45:17.368997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.368970    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-kube-api-access-h2lrh" pod "bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" (UID: "bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac"). InnerVolumeSpecName "kube-api-access-h2lrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:17.410385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.410353    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:17.410574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.410410    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2lrh\" (UniqueName: \"kubernetes.io/projected/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac-kube-api-access-h2lrh\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:17.563001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.562925    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"]
Oct 30 21:45:17.680579 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.680536    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"]
Oct 30 21:45:17.943117 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.943079    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" event={"ID":"7b6a7006-c5dc-43d4-af26-37bb73e1c593","Type":"ContainerStarted","Data":"43b192f7266d7a3437e921746a9dc1abfabe2d98b7bb70009303ac345b37d391"}
Oct 30 21:45:17.944634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.944611    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"
Oct 30 21:45:17.944797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.944655    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6" event={"ID":"bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac","Type":"ContainerDied","Data":"ed37e46107d02d4da5c9bdb28f42f6ec4b3fe09b5ce660f6d75e3c44450d6e54"}
Oct 30 21:45:17.944797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.944715    5786 scope.go:122] "RemoveContainer" containerID="c31a51362018ecb9bb399850628c9c800b6206b41607fd5a87b612b725cde3a2"
Oct 30 21:45:17.952242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:17.952201    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" event={"ID":"d325fad6-efa5-4bd8-a836-b664ec176dab","Type":"ContainerStarted","Data":"49ab40bfa421dcb3545f8944c0ab3ed992cd60305c95469b3eb2dfc4e0fd06ad"}
Oct 30 21:45:18.202686 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.202616    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"]
Oct 30 21:45:18.206498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.206471    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-815dab-29889945-tgcs6"]
Oct 30 21:45:18.352933 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.352903    5786 scope.go:122] "RemoveContainer" containerID="567e8756c96a317597cbdd429bb62ab7b6cfda90e7fda588088d08b32f41290d"
Oct 30 21:45:18.353349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.353003    5786 scope.go:122] "RemoveContainer" containerID="1b3291ce7fcbf459fb1e9de698a144adf21b2d10b272a25339abb043d3da3d56"
Oct 30 21:45:18.353349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.353086    5786 scope.go:122] "RemoveContainer" containerID="c7a9caf5161ca0da36d2ffe4e7ef32363487499e1cf88cc3da86ff37e515b0ce"
Oct 30 21:45:18.366134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.366113    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" path="/var/lib/kubelet/pods/bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac/volumes"
Oct 30 21:45:18.964399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.963731    5786 generic.go:356] "Generic (PLEG): container finished" podID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerID="49ab40bfa421dcb3545f8944c0ab3ed992cd60305c95469b3eb2dfc4e0fd06ad" exitCode=60
Oct 30 21:45:18.964399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.963757    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" event={"ID":"d325fad6-efa5-4bd8-a836-b664ec176dab","Type":"ContainerDied","Data":"49ab40bfa421dcb3545f8944c0ab3ed992cd60305c95469b3eb2dfc4e0fd06ad"}
Oct 30 21:45:18.964399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.963831    5786 scope.go:122] "RemoveContainer" containerID="45f82cf8623f683555d9e4a9d97553bd515cbbd98cede8e888870330f6353b3d"
Oct 30 21:45:18.964802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.964766    5786 scope.go:122] "RemoveContainer" containerID="49ab40bfa421dcb3545f8944c0ab3ed992cd60305c95469b3eb2dfc4e0fd06ad"
Oct 30 21:45:18.965184 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:18.965103    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f_odh-ai-gateway-infra(d325fad6-efa5-4bd8-a836-b664ec176dab)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab"
Oct 30 21:45:18.966441 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.966418    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" event={"ID":"40f37fc3-dedd-467b-8aa3-b455273e8cf3","Type":"ContainerStarted","Data":"c572c3c98f29e5912ef79cb8cacca969729b3db7de0b54196717de0d119dbfb0"}
Oct 30 21:45:18.968617 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.968599    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" event={"ID":"71da857c-d45b-4779-a63a-bd5cb8e9ba48","Type":"ContainerStarted","Data":"6edba3b52f70a7d8feed0fd8191379776fb45bd30942456f9c86308e11b50a41"}
Oct 30 21:45:18.970785 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.970764    5786 generic.go:356] "Generic (PLEG): container finished" podID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerID="43b192f7266d7a3437e921746a9dc1abfabe2d98b7bb70009303ac345b37d391" exitCode=60
Oct 30 21:45:18.970874 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.970841    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" event={"ID":"7b6a7006-c5dc-43d4-af26-37bb73e1c593","Type":"ContainerDied","Data":"43b192f7266d7a3437e921746a9dc1abfabe2d98b7bb70009303ac345b37d391"}
Oct 30 21:45:18.971356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.971340    5786 scope.go:122] "RemoveContainer" containerID="43b192f7266d7a3437e921746a9dc1abfabe2d98b7bb70009303ac345b37d391"
Oct 30 21:45:18.971626 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:18.971604    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865_odh-ai-gateway-infra(7b6a7006-c5dc-43d4-af26-37bb73e1c593)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593"
Oct 30 21:45:18.976611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:18.976594    5786 scope.go:122] "RemoveContainer" containerID="3ce16a3de7d267a134b21f2ae6fd5458ee2dff962cf17e20147899e65986787a"
Oct 30 21:45:19.127851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.127822    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"]
Oct 30 21:45:19.337281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.333060    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"]
Oct 30 21:45:19.981578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.981544    5786 generic.go:356] "Generic (PLEG): container finished" podID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerID="6edba3b52f70a7d8feed0fd8191379776fb45bd30942456f9c86308e11b50a41" exitCode=60
Oct 30 21:45:19.982039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.981584    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" event={"ID":"71da857c-d45b-4779-a63a-bd5cb8e9ba48","Type":"ContainerDied","Data":"6edba3b52f70a7d8feed0fd8191379776fb45bd30942456f9c86308e11b50a41"}
Oct 30 21:45:19.982039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.981656    5786 scope.go:122] "RemoveContainer" containerID="567e8756c96a317597cbdd429bb62ab7b6cfda90e7fda588088d08b32f41290d"
Oct 30 21:45:19.982039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.982014    5786 scope.go:122] "RemoveContainer" containerID="6edba3b52f70a7d8feed0fd8191379776fb45bd30942456f9c86308e11b50a41"
Oct 30 21:45:19.982386 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:19.982357    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx_odh-ai-gateway-infra(71da857c-d45b-4779-a63a-bd5cb8e9ba48)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48"
Oct 30 21:45:19.985727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.985707    5786 generic.go:356] "Generic (PLEG): container finished" podID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerID="81842f2b93d812c3ba00869f85441c8b98bb1440ced26f1daf013cd3dbb58fd7" exitCode=60
Oct 30 21:45:19.985844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.985735    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" event={"ID":"5695454a-c88c-48cd-aba4-207d3bbe97a3","Type":"ContainerDied","Data":"81842f2b93d812c3ba00869f85441c8b98bb1440ced26f1daf013cd3dbb58fd7"}
Oct 30 21:45:19.986123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.986105    5786 scope.go:122] "RemoveContainer" containerID="81842f2b93d812c3ba00869f85441c8b98bb1440ced26f1daf013cd3dbb58fd7"
Oct 30 21:45:19.986454 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:19.986433    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889945-2d9nv_odh-ai-gateway-infra(5695454a-c88c-48cd-aba4-207d3bbe97a3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3"
Oct 30 21:45:19.989977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.989960    5786 generic.go:356] "Generic (PLEG): container finished" podID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerID="c572c3c98f29e5912ef79cb8cacca969729b3db7de0b54196717de0d119dbfb0" exitCode=60
Oct 30 21:45:19.990083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.990037    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" event={"ID":"40f37fc3-dedd-467b-8aa3-b455273e8cf3","Type":"ContainerDied","Data":"c572c3c98f29e5912ef79cb8cacca969729b3db7de0b54196717de0d119dbfb0"}
Oct 30 21:45:19.990470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.990442    5786 scope.go:122] "RemoveContainer" containerID="c572c3c98f29e5912ef79cb8cacca969729b3db7de0b54196717de0d119dbfb0"
Oct 30 21:45:19.990767 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:19.990745    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr_odh-ai-gateway-infra(40f37fc3-dedd-467b-8aa3-b455273e8cf3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3"
Oct 30 21:45:19.995257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:19.995239    5786 scope.go:122] "RemoveContainer" containerID="c7a9caf5161ca0da36d2ffe4e7ef32363487499e1cf88cc3da86ff37e515b0ce"
Oct 30 21:45:20.007840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.007819    5786 scope.go:122] "RemoveContainer" containerID="1b3291ce7fcbf459fb1e9de698a144adf21b2d10b272a25339abb043d3da3d56"
Oct 30 21:45:20.175503 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.175466    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:20.183614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.183595    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:20.232190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.232136    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"]
Oct 30 21:45:20.267654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.267633    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d325fad6-efa5-4bd8-a836-b664ec176dab-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d325fad6-efa5-4bd8-a836-b664ec176dab-maas-api-service-ca\") pod \"d325fad6-efa5-4bd8-a836-b664ec176dab\" (UID: \"d325fad6-efa5-4bd8-a836-b664ec176dab\") "
Oct 30 21:45:20.267792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.267775    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b6a7006-c5dc-43d4-af26-37bb73e1c593-kube-api-access-r4dqr\" (UniqueName: \"kubernetes.io/projected/7b6a7006-c5dc-43d4-af26-37bb73e1c593-kube-api-access-r4dqr\") pod \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\" (UID: \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\") "
Oct 30 21:45:20.267868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.267824    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b6a7006-c5dc-43d4-af26-37bb73e1c593-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6a7006-c5dc-43d4-af26-37bb73e1c593-maas-api-service-ca\") pod \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\" (UID: \"7b6a7006-c5dc-43d4-af26-37bb73e1c593\") "
Oct 30 21:45:20.267968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.267854    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d325fad6-efa5-4bd8-a836-b664ec176dab-kube-api-access-p62nl\" (UniqueName: \"kubernetes.io/projected/d325fad6-efa5-4bd8-a836-b664ec176dab-kube-api-access-p62nl\") pod \"d325fad6-efa5-4bd8-a836-b664ec176dab\" (UID: \"d325fad6-efa5-4bd8-a836-b664ec176dab\") "
Oct 30 21:45:20.268037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.268005    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d325fad6-efa5-4bd8-a836-b664ec176dab-maas-api-service-ca" pod "d325fad6-efa5-4bd8-a836-b664ec176dab" (UID: "d325fad6-efa5-4bd8-a836-b664ec176dab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:20.268774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.268749    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b6a7006-c5dc-43d4-af26-37bb73e1c593-maas-api-service-ca" pod "7b6a7006-c5dc-43d4-af26-37bb73e1c593" (UID: "7b6a7006-c5dc-43d4-af26-37bb73e1c593"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:20.269118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.269098    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6a7006-c5dc-43d4-af26-37bb73e1c593-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:20.269169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.269125    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d325fad6-efa5-4bd8-a836-b664ec176dab-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:20.270173 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.270148    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b6a7006-c5dc-43d4-af26-37bb73e1c593-kube-api-access-r4dqr" pod "7b6a7006-c5dc-43d4-af26-37bb73e1c593" (UID: "7b6a7006-c5dc-43d4-af26-37bb73e1c593"). InnerVolumeSpecName "kube-api-access-r4dqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:20.270666 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.270645    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d325fad6-efa5-4bd8-a836-b664ec176dab-kube-api-access-p62nl" pod "d325fad6-efa5-4bd8-a836-b664ec176dab" (UID: "d325fad6-efa5-4bd8-a836-b664ec176dab"). InnerVolumeSpecName "kube-api-access-p62nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:20.350439 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.350418    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"]
Oct 30 21:45:20.370275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.370250    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4dqr\" (UniqueName: \"kubernetes.io/projected/7b6a7006-c5dc-43d4-af26-37bb73e1c593-kube-api-access-r4dqr\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:20.370275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:20.370277    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p62nl\" (UniqueName: \"kubernetes.io/projected/d325fad6-efa5-4bd8-a836-b664ec176dab-kube-api-access-p62nl\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.000114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.000090    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"
Oct 30 21:45:21.000576 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.000129    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f" event={"ID":"d325fad6-efa5-4bd8-a836-b664ec176dab","Type":"ContainerDied","Data":"4902bdc13946d3b1cd7b37011c2213d5f37572691c16f51209bb5bf59c4e8c66"}
Oct 30 21:45:21.000576 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.000199    5786 scope.go:122] "RemoveContainer" containerID="49ab40bfa421dcb3545f8944c0ab3ed992cd60305c95469b3eb2dfc4e0fd06ad"
Oct 30 21:45:21.005870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.005845    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"
Oct 30 21:45:21.005960 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.005877    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865" event={"ID":"7b6a7006-c5dc-43d4-af26-37bb73e1c593","Type":"ContainerDied","Data":"8b0aa785de3e525be29ccfbd3d235823c85219a3496eca0d745d2ab65bd5e1ad"}
Oct 30 21:45:21.011519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.011484    5786 scope.go:122] "RemoveContainer" containerID="43b192f7266d7a3437e921746a9dc1abfabe2d98b7bb70009303ac345b37d391"
Oct 30 21:45:21.182078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.182053    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:21.213507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.213490    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:21.235062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.235037    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"]
Oct 30 21:45:21.283248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.282995    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71da857c-d45b-4779-a63a-bd5cb8e9ba48-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71da857c-d45b-4779-a63a-bd5cb8e9ba48-maas-api-service-ca\") pod \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\" (UID: \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\") "
Oct 30 21:45:21.283248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.283092    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71da857c-d45b-4779-a63a-bd5cb8e9ba48-kube-api-access-zmg8m\" (UniqueName: \"kubernetes.io/projected/71da857c-d45b-4779-a63a-bd5cb8e9ba48-kube-api-access-zmg8m\") pod \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\" (UID: \"71da857c-d45b-4779-a63a-bd5cb8e9ba48\") "
Oct 30 21:45:21.283494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.283297    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40f37fc3-dedd-467b-8aa3-b455273e8cf3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f37fc3-dedd-467b-8aa3-b455273e8cf3-maas-api-service-ca\") pod \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\" (UID: \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\") "
Oct 30 21:45:21.283494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.283366    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40f37fc3-dedd-467b-8aa3-b455273e8cf3-kube-api-access-2qvmh\" (UniqueName: \"kubernetes.io/projected/40f37fc3-dedd-467b-8aa3-b455273e8cf3-kube-api-access-2qvmh\") pod \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\" (UID: \"40f37fc3-dedd-467b-8aa3-b455273e8cf3\") "
Oct 30 21:45:21.283494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.283366    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71da857c-d45b-4779-a63a-bd5cb8e9ba48-maas-api-service-ca" pod "71da857c-d45b-4779-a63a-bd5cb8e9ba48" (UID: "71da857c-d45b-4779-a63a-bd5cb8e9ba48"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:21.283676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.283613    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40f37fc3-dedd-467b-8aa3-b455273e8cf3-maas-api-service-ca" pod "40f37fc3-dedd-467b-8aa3-b455273e8cf3" (UID: "40f37fc3-dedd-467b-8aa3-b455273e8cf3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:21.284274 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.284250    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f37fc3-dedd-467b-8aa3-b455273e8cf3-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.284392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.284278    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71da857c-d45b-4779-a63a-bd5cb8e9ba48-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.286402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.286357    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f37fc3-dedd-467b-8aa3-b455273e8cf3-kube-api-access-2qvmh" pod "40f37fc3-dedd-467b-8aa3-b455273e8cf3" (UID: "40f37fc3-dedd-467b-8aa3-b455273e8cf3"). InnerVolumeSpecName "kube-api-access-2qvmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:21.286402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.286382    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"]
Oct 30 21:45:21.286590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.286459    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71da857c-d45b-4779-a63a-bd5cb8e9ba48-kube-api-access-zmg8m" pod "71da857c-d45b-4779-a63a-bd5cb8e9ba48" (UID: "71da857c-d45b-4779-a63a-bd5cb8e9ba48"). InnerVolumeSpecName "kube-api-access-zmg8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:21.290399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.290376    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-0a4eab-29889945-jq27f"]
Oct 30 21:45:21.385556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.385527    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmg8m\" (UniqueName: \"kubernetes.io/projected/71da857c-d45b-4779-a63a-bd5cb8e9ba48-kube-api-access-zmg8m\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.385556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.385555    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qvmh\" (UniqueName: \"kubernetes.io/projected/40f37fc3-dedd-467b-8aa3-b455273e8cf3-kube-api-access-2qvmh\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.396479 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.396440    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"]
Oct 30 21:45:21.400489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.400467    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99003c-29889945-fx865"]
Oct 30 21:45:21.417865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.417845    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:21.486398 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.486376    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5695454a-c88c-48cd-aba4-207d3bbe97a3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5695454a-c88c-48cd-aba4-207d3bbe97a3-maas-api-service-ca\") pod \"5695454a-c88c-48cd-aba4-207d3bbe97a3\" (UID: \"5695454a-c88c-48cd-aba4-207d3bbe97a3\") "
Oct 30 21:45:21.486514 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.486502    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5695454a-c88c-48cd-aba4-207d3bbe97a3-kube-api-access-mbpmh\" (UniqueName: \"kubernetes.io/projected/5695454a-c88c-48cd-aba4-207d3bbe97a3-kube-api-access-mbpmh\") pod \"5695454a-c88c-48cd-aba4-207d3bbe97a3\" (UID: \"5695454a-c88c-48cd-aba4-207d3bbe97a3\") "
Oct 30 21:45:21.486703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.486686    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5695454a-c88c-48cd-aba4-207d3bbe97a3-maas-api-service-ca" pod "5695454a-c88c-48cd-aba4-207d3bbe97a3" (UID: "5695454a-c88c-48cd-aba4-207d3bbe97a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:21.487032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.487011    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5695454a-c88c-48cd-aba4-207d3bbe97a3-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.488604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.488585    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5695454a-c88c-48cd-aba4-207d3bbe97a3-kube-api-access-mbpmh" pod "5695454a-c88c-48cd-aba4-207d3bbe97a3" (UID: "5695454a-c88c-48cd-aba4-207d3bbe97a3"). InnerVolumeSpecName "kube-api-access-mbpmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:21.588617 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:21.588565    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbpmh\" (UniqueName: \"kubernetes.io/projected/5695454a-c88c-48cd-aba4-207d3bbe97a3-kube-api-access-mbpmh\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:22.014596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.014569    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"
Oct 30 21:45:22.015024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.014563    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv" event={"ID":"5695454a-c88c-48cd-aba4-207d3bbe97a3","Type":"ContainerDied","Data":"c005c1bdd79c65e44664ca51dfe095e71345319fec11e4ab35a7d2eadbac4321"}
Oct 30 21:45:22.015024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.014732    5786 scope.go:122] "RemoveContainer" containerID="81842f2b93d812c3ba00869f85441c8b98bb1440ced26f1daf013cd3dbb58fd7"
Oct 30 21:45:22.017115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.017096    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"
Oct 30 21:45:22.017288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.017134    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr" event={"ID":"40f37fc3-dedd-467b-8aa3-b455273e8cf3","Type":"ContainerDied","Data":"9880b00ceff937438e20121b87e5a113744a56d0bfbcfddedd727bc36fda9ef7"}
Oct 30 21:45:22.018731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.018714    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"
Oct 30 21:45:22.018873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.018733    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx" event={"ID":"71da857c-d45b-4779-a63a-bd5cb8e9ba48","Type":"ContainerDied","Data":"f4dcf56ab3af0bda3a53b3ca890d36351f1ce17784b95d3219c2f967dd987674"}
Oct 30 21:45:22.027680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.027665    5786 scope.go:122] "RemoveContainer" containerID="c572c3c98f29e5912ef79cb8cacca969729b3db7de0b54196717de0d119dbfb0"
Oct 30 21:45:22.037889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.037871    5786 scope.go:122] "RemoveContainer" containerID="6edba3b52f70a7d8feed0fd8191379776fb45bd30942456f9c86308e11b50a41"
Oct 30 21:45:22.177685 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.177660    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"]
Oct 30 21:45:22.180962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.180944    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-fe5385-29889945-hj9vx"]
Oct 30 21:45:22.287652 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.287575    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"]
Oct 30 21:45:22.290904 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.290884    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-2d9nv"]
Oct 30 21:45:22.359473 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:22.359442    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:45:22.363856 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.363833    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" path="/var/lib/kubelet/pods/5695454a-c88c-48cd-aba4-207d3bbe97a3/volumes"
Oct 30 21:45:22.364310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.364296    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" path="/var/lib/kubelet/pods/71da857c-d45b-4779-a63a-bd5cb8e9ba48/volumes"
Oct 30 21:45:22.364740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.364727    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" path="/var/lib/kubelet/pods/7b6a7006-c5dc-43d4-af26-37bb73e1c593/volumes"
Oct 30 21:45:22.365420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.365406    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" path="/var/lib/kubelet/pods/d325fad6-efa5-4bd8-a836-b664ec176dab/volumes"
Oct 30 21:45:22.399154 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.399098    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"]
Oct 30 21:45:22.402988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.402968    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-38b832-29889945-mvbcr"]
Oct 30 21:45:22.563390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.563325    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d" podUID="c74097eb-e5f3-445c-9df6-c396f9aa9b38" containerName="payload-processing" containerID="cri-o://33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1" gracePeriod=25
Oct 30 21:45:22.681849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.681818    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s" podUID="889b5e4b-ca15-40fd-998d-bde2bcbe945c" containerName="payload-pre-processing" containerID="cri-o://8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a" gracePeriod=25
Oct 30 21:45:22.838959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.836488    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:45:22.906843 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.906781    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c74097eb-e5f3-445c-9df6-c396f9aa9b38-kube-api-access-vwbdd\" (UniqueName: \"kubernetes.io/projected/c74097eb-e5f3-445c-9df6-c396f9aa9b38-kube-api-access-vwbdd\") pod \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\" (UID: \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\") "
Oct 30 21:45:22.907013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.906945    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c74097eb-e5f3-445c-9df6-c396f9aa9b38-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c74097eb-e5f3-445c-9df6-c396f9aa9b38-plugins-config-volume\") pod \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\" (UID: \"c74097eb-e5f3-445c-9df6-c396f9aa9b38\") "
Oct 30 21:45:22.907413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.907387    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c74097eb-e5f3-445c-9df6-c396f9aa9b38-plugins-config-volume" pod "c74097eb-e5f3-445c-9df6-c396f9aa9b38" (UID: "c74097eb-e5f3-445c-9df6-c396f9aa9b38"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:22.907591 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.907576    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c74097eb-e5f3-445c-9df6-c396f9aa9b38-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:22.909301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.909269    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c74097eb-e5f3-445c-9df6-c396f9aa9b38-kube-api-access-vwbdd" pod "c74097eb-e5f3-445c-9df6-c396f9aa9b38" (UID: "c74097eb-e5f3-445c-9df6-c396f9aa9b38"). InnerVolumeSpecName "kube-api-access-vwbdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:22.931027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:22.930986    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:45:23.008427 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.008398    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/889b5e4b-ca15-40fd-998d-bde2bcbe945c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b5e4b-ca15-40fd-998d-bde2bcbe945c-plugins-config-volume\") pod \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\" (UID: \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\") "
Oct 30 21:45:23.008583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.008531    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/889b5e4b-ca15-40fd-998d-bde2bcbe945c-kube-api-access-lsp4v\" (UniqueName: \"kubernetes.io/projected/889b5e4b-ca15-40fd-998d-bde2bcbe945c-kube-api-access-lsp4v\") pod \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\" (UID: \"889b5e4b-ca15-40fd-998d-bde2bcbe945c\") "
Oct 30 21:45:23.008735 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.008713    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/889b5e4b-ca15-40fd-998d-bde2bcbe945c-plugins-config-volume" pod "889b5e4b-ca15-40fd-998d-bde2bcbe945c" (UID: "889b5e4b-ca15-40fd-998d-bde2bcbe945c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:23.009052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.009033    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b5e4b-ca15-40fd-998d-bde2bcbe945c-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:23.009110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.009057    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwbdd\" (UniqueName: \"kubernetes.io/projected/c74097eb-e5f3-445c-9df6-c396f9aa9b38-kube-api-access-vwbdd\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:23.010638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.010619    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/889b5e4b-ca15-40fd-998d-bde2bcbe945c-kube-api-access-lsp4v" pod "889b5e4b-ca15-40fd-998d-bde2bcbe945c" (UID: "889b5e4b-ca15-40fd-998d-bde2bcbe945c"). InnerVolumeSpecName "kube-api-access-lsp4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:23.031128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.031104    5786 generic.go:356] "Generic (PLEG): container finished" podID="889b5e4b-ca15-40fd-998d-bde2bcbe945c" containerID="8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a" exitCode=0
Oct 30 21:45:23.031554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.031151    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"
Oct 30 21:45:23.031554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.031150    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s" event={"ID":"889b5e4b-ca15-40fd-998d-bde2bcbe945c","Type":"ContainerDied","Data":"8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a"}
Oct 30 21:45:23.031554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.031220    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s" event={"ID":"889b5e4b-ca15-40fd-998d-bde2bcbe945c","Type":"ContainerDied","Data":"3809551bb062cb6175e2765e4c3f7769a476d4b5454fb32f5f27e99586f8f4f3"}
Oct 30 21:45:23.031554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.031252    5786 scope.go:122] "RemoveContainer" containerID="8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a"
Oct 30 21:45:23.032971 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.032948    5786 generic.go:356] "Generic (PLEG): container finished" podID="c74097eb-e5f3-445c-9df6-c396f9aa9b38" containerID="33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1" exitCode=0
Oct 30 21:45:23.033082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.032980    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"
Oct 30 21:45:23.033082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.033043    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d" event={"ID":"c74097eb-e5f3-445c-9df6-c396f9aa9b38","Type":"ContainerDied","Data":"33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1"}
Oct 30 21:45:23.033218 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.033091    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d" event={"ID":"c74097eb-e5f3-445c-9df6-c396f9aa9b38","Type":"ContainerDied","Data":"b7343269b6a1b8c2ee373f4f19770f14bb50b440ca4d2da4ceb5a992066e967d"}
Oct 30 21:45:23.044869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.044704    5786 scope.go:122] "RemoveContainer" containerID="8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a"
Oct 30 21:45:23.045030 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:23.044999    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a\": container with ID starting with 8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a not found: ID does not exist" containerID="8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a"
Oct 30 21:45:23.045105 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.045044    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a"} err="failed to get container status \"8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a\": rpc error: code = NotFound desc = could not find container \"8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a\": container with ID starting with 8fed4c7e14a3080375401f788527a70c3bb2b85080c7a379d22942802f5dbb5a not found: ID does not exist"
Oct 30 21:45:23.045105 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.045068    5786 scope.go:122] "RemoveContainer" containerID="33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1"
Oct 30 21:45:23.057603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.057557    5786 scope.go:122] "RemoveContainer" containerID="33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1"
Oct 30 21:45:23.057967 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:23.057945    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1\": container with ID starting with 33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1 not found: ID does not exist" containerID="33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1"
Oct 30 21:45:23.058082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.057972    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1"} err="failed to get container status \"33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1\": rpc error: code = NotFound desc = could not find container \"33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1\": container with ID starting with 33403f5b3ecc8cb7a3c5d71b7c74c498fecd3cfc8724cf02c7e995ac9537e3b1 not found: ID does not exist"
Oct 30 21:45:23.110754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.110695    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsp4v\" (UniqueName: \"kubernetes.io/projected/889b5e4b-ca15-40fd-998d-bde2bcbe945c-kube-api-access-lsp4v\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:23.184475 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.184446    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"]
Oct 30 21:45:23.190105 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.190084    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7cf226-7d6c7b8f55-mrq2s"]
Oct 30 21:45:23.297821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.297794    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"]
Oct 30 21:45:23.303034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:23.303012    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7cf226-64d576bf99-fvv6d"]
Oct 30 21:45:24.360027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:24.360000    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" path="/var/lib/kubelet/pods/40f37fc3-dedd-467b-8aa3-b455273e8cf3/volumes"
Oct 30 21:45:24.360441 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:24.360425    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="889b5e4b-ca15-40fd-998d-bde2bcbe945c" path="/var/lib/kubelet/pods/889b5e4b-ca15-40fd-998d-bde2bcbe945c/volumes"
Oct 30 21:45:24.360787 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:24.360773    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c74097eb-e5f3-445c-9df6-c396f9aa9b38" path="/var/lib/kubelet/pods/c74097eb-e5f3-445c-9df6-c396f9aa9b38/volumes"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.665013    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"]
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666113    5786 cpu_manager.go:414] "containerMap: removing container" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666134    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666150    5786 cpu_manager.go:414] "containerMap: removing container" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666157    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666182    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666190    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666220    5786 cpu_manager.go:414] "containerMap: removing container" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666227    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666238    5786 cpu_manager.go:414] "containerMap: removing container" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666244    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666255    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666261    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666272    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666278    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666292    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666299    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666330    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666337    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666349    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666355    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666374    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c74097eb-e5f3-445c-9df6-c396f9aa9b38" containerName="payload-processing"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666381    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c74097eb-e5f3-445c-9df6-c396f9aa9b38" containerName="payload-processing"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666407    5786 cpu_manager.go:414] "containerMap: removing container" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666415    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666436    5786 cpu_manager.go:414] "containerMap: removing container" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666443    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666453    5786 cpu_manager.go:414] "containerMap: removing container" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666459    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666478    5786 cpu_manager.go:414] "containerMap: removing container" podUID="889b5e4b-ca15-40fd-998d-bde2bcbe945c" containerName="payload-pre-processing"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666485    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="889b5e4b-ca15-40fd-998d-bde2bcbe945c" containerName="payload-pre-processing"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666668    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666682    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666692    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666703    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666713    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666722    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666732    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666740    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c74097eb-e5f3-445c-9df6-c396f9aa9b38" containerName="payload-processing"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666750    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666758    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666767    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666776    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="889b5e4b-ca15-40fd-998d-bde2bcbe945c" containerName="payload-pre-processing"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666786    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666794    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666804    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.673236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.666813    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.697229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.696522    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:27.730229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.728599    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"]
Oct 30 21:45:27.762111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.761480    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-3ab88248\""
Oct 30 21:45:27.766520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.766477    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-3ab88248-dockercfg-m6vqb\""
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.778744    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"]
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780036    5786 cpu_manager.go:414] "containerMap: removing container" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780060    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcf6a8a8-b9f1-4d7b-9de9-29f3fc99d7ac" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780080    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780087    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b6a7006-c5dc-43d4-af26-37bb73e1c593" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780104    5786 cpu_manager.go:414] "containerMap: removing container" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780111    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780127    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780134    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780170    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780178    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780383    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="40f37fc3-dedd-467b-8aa3-b455273e8cf3" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780403    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="71da857c-d45b-4779-a63a-bd5cb8e9ba48" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780414    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d325fad6-efa5-4bd8-a836-b664ec176dab" containerName="cleanup"
Oct 30 21:45:27.781228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.780425    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5695454a-c88c-48cd-aba4-207d3bbe97a3" containerName="cleanup"
Oct 30 21:45:27.786292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.785018    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:27.789257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.789226    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pjbh\" (UniqueName: \"kubernetes.io/projected/903ce8c0-dd05-491c-a34f-344fd7275b31-kube-api-access-4pjbh\") pod \"payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b\" (UID: \"903ce8c0-dd05-491c-a34f-344fd7275b31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:27.789530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.789513    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/903ce8c0-dd05-491c-a34f-344fd7275b31-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b\" (UID: \"903ce8c0-dd05-491c-a34f-344fd7275b31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:27.844470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.844442    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"]
Oct 30 21:45:27.890969 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.890945    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pjbh\" (UniqueName: \"kubernetes.io/projected/903ce8c0-dd05-491c-a34f-344fd7275b31-kube-api-access-4pjbh\") pod \"payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b\" (UID: \"903ce8c0-dd05-491c-a34f-344fd7275b31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:27.891108 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.890990    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/903ce8c0-dd05-491c-a34f-344fd7275b31-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b\" (UID: \"903ce8c0-dd05-491c-a34f-344fd7275b31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:27.891108 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.891061    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00878b67-261a-4b77-ac91-a29de49126ae-plugins-config-volume\") pod \"payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58\" (UID: \"00878b67-261a-4b77-ac91-a29de49126ae\") " pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:27.891191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.891140    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvr94\" (UniqueName: \"kubernetes.io/projected/00878b67-261a-4b77-ac91-a29de49126ae-kube-api-access-gvr94\") pod \"payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58\" (UID: \"00878b67-261a-4b77-ac91-a29de49126ae\") " pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:27.891647 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.891630    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/903ce8c0-dd05-491c-a34f-344fd7275b31-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b\" (UID: \"903ce8c0-dd05-491c-a34f-344fd7275b31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:27.952773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.952702    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pjbh\" (UniqueName: \"kubernetes.io/projected/903ce8c0-dd05-491c-a34f-344fd7275b31-kube-api-access-4pjbh\") pod \"payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b\" (UID: \"903ce8c0-dd05-491c-a34f-344fd7275b31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:27.992665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.992637    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00878b67-261a-4b77-ac91-a29de49126ae-plugins-config-volume\") pod \"payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58\" (UID: \"00878b67-261a-4b77-ac91-a29de49126ae\") " pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:27.992803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.992694    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvr94\" (UniqueName: \"kubernetes.io/projected/00878b67-261a-4b77-ac91-a29de49126ae-kube-api-access-gvr94\") pod \"payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58\" (UID: \"00878b67-261a-4b77-ac91-a29de49126ae\") " pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:27.993291 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:27.993274    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00878b67-261a-4b77-ac91-a29de49126ae-plugins-config-volume\") pod \"payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58\" (UID: \"00878b67-261a-4b77-ac91-a29de49126ae\") " pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:28.049263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.049229    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvr94\" (UniqueName: \"kubernetes.io/projected/00878b67-261a-4b77-ac91-a29de49126ae-kube-api-access-gvr94\") pod \"payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58\" (UID: \"00878b67-261a-4b77-ac91-a29de49126ae\") " pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:28.053016 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.052997    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:28.108486 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.108452    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"]
Oct 30 21:45:28.108633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.108549    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:28.113024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.113002    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.173250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.166586    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:45:28.173250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.166695    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:45:28.173250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.172854    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"]
Oct 30 21:45:28.196760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.196730    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.196932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.196913    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.197117 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.197093    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqdxg\" (UniqueName: \"kubernetes.io/projected/38f2fbc5-0038-4309-b84b-890c561871cf-kube-api-access-qqdxg\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.298056 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.298025    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqdxg\" (UniqueName: \"kubernetes.io/projected/38f2fbc5-0038-4309-b84b-890c561871cf-kube-api-access-qqdxg\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.298253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.298127    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.298253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.298248    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.298771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.298750    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.300583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.300566    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.354159 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.354133    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqdxg\" (UniqueName: \"kubernetes.io/projected/38f2fbc5-0038-4309-b84b-890c561871cf-kube-api-access-qqdxg\") pod \"maas-api-revoke-sub-66b312e3-zmcbp\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.432155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.432125    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"]
Oct 30 21:45:28.433950 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:45:28.433928    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod903ce8c0_dd05_491c_a34f_344fd7275b31.slice/crio-8e0cc54c8f7be396d87e4754acceee502098a20e4c220f792ae937b3e255fb64 WatchSource:0}: Error finding container 8e0cc54c8f7be396d87e4754acceee502098a20e4c220f792ae937b3e255fb64: Status 404 returned error can't find the container with id 8e0cc54c8f7be396d87e4754acceee502098a20e4c220f792ae937b3e255fb64
Oct 30 21:45:28.459194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.459140    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:28.559243 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.558663    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"]
Oct 30 21:45:28.853432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:28.853406    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"]
Oct 30 21:45:28.854063 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:45:28.854040    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f2fbc5_0038_4309_b84b_890c561871cf.slice/crio-f22a10f9d4ef5018f0a2f96c3bae5bb26179cf9da2ec9d4dceeebe23ff5d5077 WatchSource:0}: Error finding container f22a10f9d4ef5018f0a2f96c3bae5bb26179cf9da2ec9d4dceeebe23ff5d5077: Status 404 returned error can't find the container with id f22a10f9d4ef5018f0a2f96c3bae5bb26179cf9da2ec9d4dceeebe23ff5d5077
Oct 30 21:45:29.086296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.086267    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b" event={"ID":"903ce8c0-dd05-491c-a34f-344fd7275b31","Type":"ContainerStarted","Data":"c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234"}
Oct 30 21:45:29.086464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.086325    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b" event={"ID":"903ce8c0-dd05-491c-a34f-344fd7275b31","Type":"ContainerStarted","Data":"8e0cc54c8f7be396d87e4754acceee502098a20e4c220f792ae937b3e255fb64"}
Oct 30 21:45:29.086464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.086364    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:29.087823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.087802    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp" event={"ID":"38f2fbc5-0038-4309-b84b-890c561871cf","Type":"ContainerStarted","Data":"f22a10f9d4ef5018f0a2f96c3bae5bb26179cf9da2ec9d4dceeebe23ff5d5077"}
Oct 30 21:45:29.089354 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.089332    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58" event={"ID":"00878b67-261a-4b77-ac91-a29de49126ae","Type":"ContainerStarted","Data":"d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6"}
Oct 30 21:45:29.089453 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.089369    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58" event={"ID":"00878b67-261a-4b77-ac91-a29de49126ae","Type":"ContainerStarted","Data":"076ec678c578cc42695c13f8b51447fbbbf9302915ef32f5440641fe0b4b93e1"}
Oct 30 21:45:29.089453 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.089407    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:29.200625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.200585    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b" podStartSLOduration=2.200570834 podStartE2EDuration="2.200570834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:29.199698297 +0000 UTC m=+1985.070817417" watchObservedRunningTime="2026-10-30 21:45:29.200570834 +0000 UTC m=+1985.071689992"
Oct 30 21:45:29.356462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:29.355182    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58" podStartSLOduration=2.355155373 podStartE2EDuration="2.355155373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:29.328729202 +0000 UTC m=+1985.199848322" watchObservedRunningTime="2026-10-30 21:45:29.355155373 +0000 UTC m=+1985.226274493"
Oct 30 21:45:30.098379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:30.098304    5786 generic.go:356] "Generic (PLEG): container finished" podID="38f2fbc5-0038-4309-b84b-890c561871cf" containerID="f4b3779e9ebb7014cf933981b1760943d50ae20636d8d68c033e83434e437f6d" exitCode=0
Oct 30 21:45:30.098726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:30.098389    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp" event={"ID":"38f2fbc5-0038-4309-b84b-890c561871cf","Type":"ContainerDied","Data":"f4b3779e9ebb7014cf933981b1760943d50ae20636d8d68c033e83434e437f6d"}
Oct 30 21:45:31.322894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.322872    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:31.498280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.498198    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-openshift-ca\") pod \"38f2fbc5-0038-4309-b84b-890c561871cf\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") "
Oct 30 21:45:31.498280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.498259    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-xks-ca\") pod \"38f2fbc5-0038-4309-b84b-890c561871cf\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") "
Oct 30 21:45:31.498517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.498342    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38f2fbc5-0038-4309-b84b-890c561871cf-kube-api-access-qqdxg\" (UniqueName: \"kubernetes.io/projected/38f2fbc5-0038-4309-b84b-890c561871cf-kube-api-access-qqdxg\") pod \"38f2fbc5-0038-4309-b84b-890c561871cf\" (UID: \"38f2fbc5-0038-4309-b84b-890c561871cf\") "
Oct 30 21:45:31.498608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.498572    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-openshift-ca" pod "38f2fbc5-0038-4309-b84b-890c561871cf" (UID: "38f2fbc5-0038-4309-b84b-890c561871cf"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:31.499614 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.499578    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:31.500690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.500664    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-xks-ca" pod "38f2fbc5-0038-4309-b84b-890c561871cf" (UID: "38f2fbc5-0038-4309-b84b-890c561871cf"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:45:31.500788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.500747    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f2fbc5-0038-4309-b84b-890c561871cf-kube-api-access-qqdxg" pod "38f2fbc5-0038-4309-b84b-890c561871cf" (UID: "38f2fbc5-0038-4309-b84b-890c561871cf"). InnerVolumeSpecName "kube-api-access-qqdxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:31.601161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.601131    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f2fbc5-0038-4309-b84b-890c561871cf-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:31.601161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:31.601157    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqdxg\" (UniqueName: \"kubernetes.io/projected/38f2fbc5-0038-4309-b84b-890c561871cf-kube-api-access-qqdxg\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:32.180346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:32.180320    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"
Oct 30 21:45:32.180522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:32.180356    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp" event={"ID":"38f2fbc5-0038-4309-b84b-890c561871cf","Type":"ContainerDied","Data":"f22a10f9d4ef5018f0a2f96c3bae5bb26179cf9da2ec9d4dceeebe23ff5d5077"}
Oct 30 21:45:32.180522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:32.180400    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f22a10f9d4ef5018f0a2f96c3bae5bb26179cf9da2ec9d4dceeebe23ff5d5077"
Oct 30 21:45:39.175875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.175836    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"]
Oct 30 21:45:39.176570 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.176555    5786 cpu_manager.go:414] "containerMap: removing container" podUID="38f2fbc5-0038-4309-b84b-890c561871cf" containerName="revoke-keys"
Oct 30 21:45:39.176628 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.176572    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38f2fbc5-0038-4309-b84b-890c561871cf" containerName="revoke-keys"
Oct 30 21:45:39.176729 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.176719    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="38f2fbc5-0038-4309-b84b-890c561871cf" containerName="revoke-keys"
Oct 30 21:45:39.179342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.179327    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.246455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.246423    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/39e9863c-1b01-437b-b794-130f4464a923-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.246625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.246605    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/39e9863c-1b01-437b-b794-130f4464a923-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.246814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.246799    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd9vp\" (UniqueName: \"kubernetes.io/projected/39e9863c-1b01-437b-b794-130f4464a923-kube-api-access-zd9vp\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.281565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.281539    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:45:39.281726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.281668    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:45:39.285667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.285644    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"]
Oct 30 21:45:39.358229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.349616    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd9vp\" (UniqueName: \"kubernetes.io/projected/39e9863c-1b01-437b-b794-130f4464a923-kube-api-access-zd9vp\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.358229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.349693    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/39e9863c-1b01-437b-b794-130f4464a923-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.358229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.349823    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/39e9863c-1b01-437b-b794-130f4464a923-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.358229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.353144    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/39e9863c-1b01-437b-b794-130f4464a923-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.358229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.353230    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/39e9863c-1b01-437b-b794-130f4464a923-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.409646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.409619    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd9vp\" (UniqueName: \"kubernetes.io/projected/39e9863c-1b01-437b-b794-130f4464a923-kube-api-access-zd9vp\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.493296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.493229    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:39.917751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.910173    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"]
Oct 30 21:45:39.917751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.910488    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy" containerID="cri-o://27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f" gracePeriod=30
Oct 30 21:45:39.938604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:39.936111    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"]
Oct 30 21:45:40.153315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:40.149942    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:45:40.171355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:40.170352    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:45:40.279418 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:40.279384    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g" event={"ID":"39e9863c-1b01-437b-b794-130f4464a923","Type":"ContainerStarted","Data":"9af6b1c4059951f84771af01d8a4c96fbaad8ae16bf30c2c79f3ee3ec9dc7888"}
Oct 30 21:45:40.803244 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:40.803142    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"]
Oct 30 21:45:40.807950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:40.807925    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:40.870235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:40.863531    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"]
Oct 30 21:45:41.002232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.002089    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qgbw\" (UniqueName: \"kubernetes.io/projected/6597ed43-c6f1-4b6b-b111-24557a9db736-kube-api-access-7qgbw\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.002436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.002248    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.002539 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.002454    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.119180 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.119073    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qgbw\" (UniqueName: \"kubernetes.io/projected/6597ed43-c6f1-4b6b-b111-24557a9db736-kube-api-access-7qgbw\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.119343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.119193    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.119404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.119389    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.120260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.120198    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.161682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.161572    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.217229 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.211006    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qgbw\" (UniqueName: \"kubernetes.io/projected/6597ed43-c6f1-4b6b-b111-24557a9db736-kube-api-access-7qgbw\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.320245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.319429    5786 generic.go:356] "Generic (PLEG): container finished" podID="39e9863c-1b01-437b-b794-130f4464a923" containerID="6f5d6ddd2b28091ed83a38868949b4252348f927478030051a7a0ce41d111851" exitCode=0
Oct 30 21:45:41.320245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.319618    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g" event={"ID":"39e9863c-1b01-437b-b794-130f4464a923","Type":"ContainerDied","Data":"6f5d6ddd2b28091ed83a38868949b4252348f927478030051a7a0ce41d111851"}
Oct 30 21:45:41.436721 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.431758    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:41.792505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.792384    5786 patch_prober.go:29] interesting pod/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:45:41.795636 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:41.795587    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:45:42.612241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:42.600756    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"]
Oct 30 21:45:42.976259 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:42.964690    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:43.021232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.020934    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:43.065232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.056840    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:43.359950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.359924    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:43.478167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.478134    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39e9863c-1b01-437b-b794-130f4464a923-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/39e9863c-1b01-437b-b794-130f4464a923-maas-api-xks-ca\") pod \"39e9863c-1b01-437b-b794-130f4464a923\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") "
Oct 30 21:45:43.478342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.478305    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39e9863c-1b01-437b-b794-130f4464a923-kube-api-access-zd9vp\" (UniqueName: \"kubernetes.io/projected/39e9863c-1b01-437b-b794-130f4464a923-kube-api-access-zd9vp\") pod \"39e9863c-1b01-437b-b794-130f4464a923\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") "
Oct 30 21:45:43.478537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.478517    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39e9863c-1b01-437b-b794-130f4464a923-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/39e9863c-1b01-437b-b794-130f4464a923-maas-api-openshift-ca\") pod \"39e9863c-1b01-437b-b794-130f4464a923\" (UID: \"39e9863c-1b01-437b-b794-130f4464a923\") "
Oct 30 21:45:43.479262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.479236    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e9863c-1b01-437b-b794-130f4464a923-maas-api-openshift-ca" pod "39e9863c-1b01-437b-b794-130f4464a923" (UID: "39e9863c-1b01-437b-b794-130f4464a923"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:43.480707 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.480683    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e9863c-1b01-437b-b794-130f4464a923-maas-api-xks-ca" pod "39e9863c-1b01-437b-b794-130f4464a923" (UID: "39e9863c-1b01-437b-b794-130f4464a923"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:45:43.480957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.480913    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e9863c-1b01-437b-b794-130f4464a923-kube-api-access-zd9vp" pod "39e9863c-1b01-437b-b794-130f4464a923" (UID: "39e9863c-1b01-437b-b794-130f4464a923"). InnerVolumeSpecName "kube-api-access-zd9vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:43.497224 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.497179    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7" event={"ID":"6597ed43-c6f1-4b6b-b111-24557a9db736","Type":"ContainerStarted","Data":"56655509c7c9752fe0c1453e7a7ab491ce3a0233367c6b4395b80cd57bd5cd82"}
Oct 30 21:45:43.497380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.497316    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7" event={"ID":"6597ed43-c6f1-4b6b-b111-24557a9db736","Type":"ContainerStarted","Data":"9468af492e0c8d779988676db83e218564943d268114300cb56338b4ddcb1f3e"}
Oct 30 21:45:43.498634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.498615    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"
Oct 30 21:45:43.498755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.498659    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g" event={"ID":"39e9863c-1b01-437b-b794-130f4464a923","Type":"ContainerDied","Data":"9af6b1c4059951f84771af01d8a4c96fbaad8ae16bf30c2c79f3ee3ec9dc7888"}
Oct 30 21:45:43.498755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.498695    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9af6b1c4059951f84771af01d8a4c96fbaad8ae16bf30c2c79f3ee3ec9dc7888"
Oct 30 21:45:43.580881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.580816    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd9vp\" (UniqueName: \"kubernetes.io/projected/39e9863c-1b01-437b-b794-130f4464a923-kube-api-access-zd9vp\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:43.580970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.580923    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/39e9863c-1b01-437b-b794-130f4464a923-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:43.580970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.580945    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/39e9863c-1b01-437b-b794-130f4464a923-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:43.613593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:43.613555    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7" podStartSLOduration=3.613542679 podStartE2EDuration="3.613542679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:43.610320366 +0000 UTC m=+1999.481439508" watchObservedRunningTime="2026-10-30 21:45:43.613542679 +0000 UTC m=+1999.484661797"
Oct 30 21:45:44.405690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:44.405601    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"]
Oct 30 21:45:44.405867 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:44.405818    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy" containerID="cri-o://8f315b550478de98765c2ac8428262608eb78ad8bad9251acf2c5e824e73541c" gracePeriod=30
Oct 30 21:45:44.455272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:44.455241    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"]
Oct 30 21:45:44.455492 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:44.455471    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy" containerID="cri-o://34eec1c089350bd5e6d42e9d43af398ac9988be86d48b7af6424a175952d03a9" gracePeriod=30
Oct 30 21:45:44.508387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:44.508360    5786 generic.go:356] "Generic (PLEG): container finished" podID="6597ed43-c6f1-4b6b-b111-24557a9db736" containerID="56655509c7c9752fe0c1453e7a7ab491ce3a0233367c6b4395b80cd57bd5cd82" exitCode=0
Oct 30 21:45:44.508557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:44.508445    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7" event={"ID":"6597ed43-c6f1-4b6b-b111-24557a9db736","Type":"ContainerDied","Data":"56655509c7c9752fe0c1453e7a7ab491ce3a0233367c6b4395b80cd57bd5cd82"}
Oct 30 21:45:45.206516 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.206457    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:45:45.302810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.302777    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-socket\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.302989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.302841    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-certs\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.302989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.302884    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-credential-socket\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.302989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.302926    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-data\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.302989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.302980    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-kube-api-access-6lgdr\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-kube-api-access-6lgdr\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.303159 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303076    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-envoy\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.303159 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303076    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-socket" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:45.303159 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303125    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-token\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-token\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.303359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303167    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-certs" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:45.303359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303192    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-credential-socket" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:45.303359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303271    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-data" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:45.303359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303268    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30a4fbd5-4b68-4316-8e2b-c693757f3009-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30a4fbd5-4b68-4316-8e2b-c693757f3009-istiod-ca-cert\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.303359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303328    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-podinfo\") pod \"30a4fbd5-4b68-4316-8e2b-c693757f3009\" (UID: \"30a4fbd5-4b68-4316-8e2b-c693757f3009\") "
Oct 30 21:45:45.303643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303574    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a4fbd5-4b68-4316-8e2b-c693757f3009-istiod-ca-cert" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:45.304006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.303984    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.304109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.304006    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.304109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.304020    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.304109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.304032    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.304109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.304047    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30a4fbd5-4b68-4316-8e2b-c693757f3009-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.305431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.305406    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-token" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:45.305527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.305506    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-kube-api-access-6lgdr" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "kube-api-access-6lgdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:45.305915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.305892    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-envoy" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:45.305915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.305902    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-podinfo" pod "30a4fbd5-4b68-4316-8e2b-c693757f3009" (UID: "30a4fbd5-4b68-4316-8e2b-c693757f3009"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:45:45.405841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.405811    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lgdr\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-kube-api-access-6lgdr\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.405841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.405837    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.405841 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.405847    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.406066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.405855    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30a4fbd5-4b68-4316-8e2b-c693757f3009-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.518115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.518039    5786 generic.go:356] "Generic (PLEG): container finished" podID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerID="27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f" exitCode=0
Oct 30 21:45:45.518115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.518093    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"
Oct 30 21:45:45.518344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.518133    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" event={"ID":"30a4fbd5-4b68-4316-8e2b-c693757f3009","Type":"ContainerDied","Data":"27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f"}
Oct 30 21:45:45.518344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.518194    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd" event={"ID":"30a4fbd5-4b68-4316-8e2b-c693757f3009","Type":"ContainerDied","Data":"fc8fe615a16f2dfc4613dfc11c7b6d5962ed65830b8445b28ee1020875d2eaf7"}
Oct 30 21:45:45.518344 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.518228    5786 scope.go:122] "RemoveContainer" containerID="27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f"
Oct 30 21:45:45.529943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.529927    5786 scope.go:122] "RemoveContainer" containerID="862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033"
Oct 30 21:45:45.541522 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.541505    5786 scope.go:122] "RemoveContainer" containerID="27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f"
Oct 30 21:45:45.541844 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:45.541825    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f\": container with ID starting with 27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f not found: ID does not exist" containerID="27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f"
Oct 30 21:45:45.541912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.541851    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f"} err="failed to get container status \"27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f\": rpc error: code = NotFound desc = could not find container \"27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f\": container with ID starting with 27c326a33a11a7c12c0a38d44de132d9ef025df6a68dd5bb70f1a8ac47f1748f not found: ID does not exist"
Oct 30 21:45:45.541912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.541869    5786 scope.go:122] "RemoveContainer" containerID="862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033"
Oct 30 21:45:45.542137 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:45.542120    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033\": container with ID starting with 862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033 not found: ID does not exist" containerID="862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033"
Oct 30 21:45:45.542175 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.542145    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033"} err="failed to get container status \"862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033\": rpc error: code = NotFound desc = could not find container \"862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033\": container with ID starting with 862e80ee2de801e9c39a689f59160ee894fe86700c698ed4d828d9b23b250033 not found: ID does not exist"
Oct 30 21:45:45.649416 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.649391    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:45.681781 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.681756    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"]
Oct 30 21:45:45.684990 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.684972    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7cf226-openshift-default-7795f55cc7-srsqd"]
Oct 30 21:45:45.810710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.810640    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-xks-ca\") pod \"6597ed43-c6f1-4b6b-b111-24557a9db736\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") "
Oct 30 21:45:45.810860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.810749    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6597ed43-c6f1-4b6b-b111-24557a9db736-kube-api-access-7qgbw\" (UniqueName: \"kubernetes.io/projected/6597ed43-c6f1-4b6b-b111-24557a9db736-kube-api-access-7qgbw\") pod \"6597ed43-c6f1-4b6b-b111-24557a9db736\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") "
Oct 30 21:45:45.810860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.810818    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-openshift-ca\") pod \"6597ed43-c6f1-4b6b-b111-24557a9db736\" (UID: \"6597ed43-c6f1-4b6b-b111-24557a9db736\") "
Oct 30 21:45:45.811129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.811104    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-openshift-ca" pod "6597ed43-c6f1-4b6b-b111-24557a9db736" (UID: "6597ed43-c6f1-4b6b-b111-24557a9db736"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:45.811624 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.811520    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.812936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.812909    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-xks-ca" pod "6597ed43-c6f1-4b6b-b111-24557a9db736" (UID: "6597ed43-c6f1-4b6b-b111-24557a9db736"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:45:45.813041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.812965    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6597ed43-c6f1-4b6b-b111-24557a9db736-kube-api-access-7qgbw" pod "6597ed43-c6f1-4b6b-b111-24557a9db736" (UID: "6597ed43-c6f1-4b6b-b111-24557a9db736"). InnerVolumeSpecName "kube-api-access-7qgbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:45.912855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.912829    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6597ed43-c6f1-4b6b-b111-24557a9db736-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:45.912855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:45.912852    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qgbw\" (UniqueName: \"kubernetes.io/projected/6597ed43-c6f1-4b6b-b111-24557a9db736-kube-api-access-7qgbw\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:46.361027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:46.360997    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" path="/var/lib/kubelet/pods/30a4fbd5-4b68-4316-8e2b-c693757f3009/volumes"
Oct 30 21:45:46.527945 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:46.527922    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"
Oct 30 21:45:46.527945 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:46.527937    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7" event={"ID":"6597ed43-c6f1-4b6b-b111-24557a9db736","Type":"ContainerDied","Data":"9468af492e0c8d779988676db83e218564943d268114300cb56338b4ddcb1f3e"}
Oct 30 21:45:46.528135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:46.527996    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9468af492e0c8d779988676db83e218564943d268114300cb56338b4ddcb1f3e"
Oct 30 21:45:49.419894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.419859    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-v64g9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": read tcp 10.134.0.2:54122->10.134.0.141:15021: read: connection reset by peer" start-of-body=
Oct 30 21:45:49.420392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.419931    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": read tcp 10.134.0.2:54122->10.134.0.141:15021: read: connection reset by peer"
Oct 30 21:45:49.586810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.586774    5786 generic.go:356] "Generic (PLEG): container finished" podID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerID="34eec1c089350bd5e6d42e9d43af398ac9988be86d48b7af6424a175952d03a9" exitCode=0
Oct 30 21:45:49.587002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.586814    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" event={"ID":"03fdf533-e9a8-4410-910b-dfff5aaa73fb","Type":"ContainerDied","Data":"34eec1c089350bd5e6d42e9d43af398ac9988be86d48b7af6424a175952d03a9"}
Oct 30 21:45:49.588868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.588837    5786 generic.go:356] "Generic (PLEG): container finished" podID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerID="8f315b550478de98765c2ac8428262608eb78ad8bad9251acf2c5e824e73541c" exitCode=0
Oct 30 21:45:49.588993 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.588865    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" event={"ID":"6567df37-bf65-475c-bc04-d7f9c68667ad","Type":"ContainerDied","Data":"8f315b550478de98765c2ac8428262608eb78ad8bad9251acf2c5e824e73541c"}
Oct 30 21:45:49.770281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.770245    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:45:49.793915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.793894    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:45:49.806432 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806407    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-socket\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.806556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806466    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istiod-ca-cert\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.806556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806539    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-credential-socket\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.806674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806599    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-kube-api-access-txgkn\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-kube-api-access-txgkn\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.806674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806659    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-socket" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.806781 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806758    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-certs\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.806838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806784    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-credential-socket" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.806838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806790    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-podinfo\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.806838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806817    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-token\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-token\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.806838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806821    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istiod-ca-cert" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:49.807039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806850    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-socket\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.807039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806881    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-token\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-token\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.807039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806915    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-podinfo\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.807039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806962    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-certs\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.807039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.806978    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-certs" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.807039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807017    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-data\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807055    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-envoy\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807081    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-socket" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807104    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-data\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807132    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-credential-socket\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807162    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6567df37-bf65-475c-bc04-d7f9c68667ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6567df37-bf65-475c-bc04-d7f9c68667ad-istiod-ca-cert\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807199    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-envoy\") pod \"6567df37-bf65-475c-bc04-d7f9c68667ad\" (UID: \"6567df37-bf65-475c-bc04-d7f9c68667ad\") "
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807264    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-certs" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807294    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-kube-api-access-2kw2x\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-kube-api-access-2kw2x\") pod \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\" (UID: \"03fdf533-e9a8-4410-910b-dfff5aaa73fb\") "
Oct 30 21:45:49.807385 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807341    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-data" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.807854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.807483    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-data" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.808171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808146    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.808295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808175    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.808295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808191    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.808295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808220    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.808295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808235    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.808295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808248    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.808295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808262    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.808295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808275    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.808779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.808756    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6567df37-bf65-475c-bc04-d7f9c68667ad-istiod-ca-cert" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:49.809026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.809003    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-credential-socket" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.809608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.809581    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-token" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:49.809720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.809604    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-kube-api-access-txgkn" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "kube-api-access-txgkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:49.809720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.809657    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-token" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:49.809842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.809793    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-kube-api-access-2kw2x" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "kube-api-access-2kw2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:49.810090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.810074    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-envoy" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.810243 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.810221    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-podinfo" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:45:49.810712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.810694    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-podinfo" pod "03fdf533-e9a8-4410-910b-dfff5aaa73fb" (UID: "03fdf533-e9a8-4410-910b-dfff5aaa73fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:45:49.811227 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.811196    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-envoy" pod "6567df37-bf65-475c-bc04-d7f9c68667ad" (UID: "6567df37-bf65-475c-bc04-d7f9c68667ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913359    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913397    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913412    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913427    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913441    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913453    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fdf533-e9a8-4410-910b-dfff5aaa73fb-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913463    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6567df37-bf65-475c-bc04-d7f9c68667ad-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913476    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6567df37-bf65-475c-bc04-d7f9c68667ad-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913491    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kw2x\" (UniqueName: \"kubernetes.io/projected/03fdf533-e9a8-4410-910b-dfff5aaa73fb-kube-api-access-2kw2x\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.917242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.913507    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txgkn\" (UniqueName: \"kubernetes.io/projected/6567df37-bf65-475c-bc04-d7f9c68667ad-kube-api-access-txgkn\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:49.962232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.959788    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"]
Oct 30 21:45:49.971166 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:49.970970    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-0a4eab-tj27t"]
Oct 30 21:45:50.445632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.444084    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b22a470d-c6d8-4088-9f80-bf270cd5233d" path="/var/lib/kubelet/pods/b22a470d-c6d8-4088-9f80-bf270cd5233d/volumes"
Oct 30 21:45:50.509326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.508409    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"]
Oct 30 21:45:50.510830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510793    5786 cpu_manager.go:414] "containerMap: removing container" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy"
Oct 30 21:45:50.510830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510823    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy"
Oct 30 21:45:50.511000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510846    5786 cpu_manager.go:414] "containerMap: removing container" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy"
Oct 30 21:45:50.511000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510854    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy"
Oct 30 21:45:50.511000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510874    5786 cpu_manager.go:414] "containerMap: removing container" podUID="6597ed43-c6f1-4b6b-b111-24557a9db736" containerName="revoke-keys"
Oct 30 21:45:50.511000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510882    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6597ed43-c6f1-4b6b-b111-24557a9db736" containerName="revoke-keys"
Oct 30 21:45:50.511000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510929    5786 cpu_manager.go:414] "containerMap: removing container" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy"
Oct 30 21:45:50.511000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510937    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy"
Oct 30 21:45:50.511000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510958    5786 cpu_manager.go:414] "containerMap: removing container" podUID="39e9863c-1b01-437b-b794-130f4464a923" containerName="revoke-keys"
Oct 30 21:45:50.511000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.510966    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39e9863c-1b01-437b-b794-130f4464a923" containerName="revoke-keys"
Oct 30 21:45:50.511410 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.511390    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy"
Oct 30 21:45:50.511466 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.511429    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" containerName="istio-proxy"
Oct 30 21:45:50.511466 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.511450    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy"
Oct 30 21:45:50.511561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.511466    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="6597ed43-c6f1-4b6b-b111-24557a9db736" containerName="revoke-keys"
Oct 30 21:45:50.511561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.511479    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" containerName="istio-proxy"
Oct 30 21:45:50.511561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.511496    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="39e9863c-1b01-437b-b794-130f4464a923" containerName="revoke-keys"
Oct 30 21:45:50.521187 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.521161    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"]
Oct 30 21:45:50.521364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.521305    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:50.523481 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.523458    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"]
Oct 30 21:45:50.524553 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.524536    5786 cpu_manager.go:414] "containerMap: removing container" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy"
Oct 30 21:45:50.524553 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.524553    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30a4fbd5-4b68-4316-8e2b-c693757f3009" containerName="istio-proxy"
Oct 30 21:45:50.527595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.527580    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:50.542469 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.542443    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"]
Oct 30 21:45:50.572901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.572872    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-9085f4\""
Oct 30 21:45:50.597031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.597010    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"
Oct 30 21:45:50.597149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.597026    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5" event={"ID":"03fdf533-e9a8-4410-910b-dfff5aaa73fb","Type":"ContainerDied","Data":"f9f7b98999ef4e569b201d0d70456e6d5b40e40e30346975496d0ceb3217af6a"}
Oct 30 21:45:50.597149 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.597079    5786 scope.go:122] "RemoveContainer" containerID="34eec1c089350bd5e6d42e9d43af398ac9988be86d48b7af6424a175952d03a9"
Oct 30 21:45:50.598799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.598781    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9" event={"ID":"6567df37-bf65-475c-bc04-d7f9c68667ad","Type":"ContainerDied","Data":"64e5323e9b1382a28c3385ede05f349059e54571776d97e829821a01b7b75c15"}
Oct 30 21:45:50.598878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.598867    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"
Oct 30 21:45:50.607722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.607702    5786 scope.go:122] "RemoveContainer" containerID="8f315b550478de98765c2ac8428262608eb78ad8bad9251acf2c5e824e73541c"
Oct 30 21:45:50.655360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.655339    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-9085f4-dockercfg-gm4ds\""
Oct 30 21:45:50.694228 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.694166    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr59q\" (UniqueName: \"kubernetes.io/projected/80159d30-c714-496e-b901-38bb4ce46d8d-kube-api-access-rr59q\") pod \"payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk\" (UID: \"80159d30-c714-496e-b901-38bb4ce46d8d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:50.694346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.694328    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3fc218-76e9-41e6-80be-1b44f16d4110-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m\" (UID: \"9c3fc218-76e9-41e6-80be-1b44f16d4110\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:50.694444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.694431    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7swf\" (UniqueName: \"kubernetes.io/projected/9c3fc218-76e9-41e6-80be-1b44f16d4110-kube-api-access-m7swf\") pod \"payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m\" (UID: \"9c3fc218-76e9-41e6-80be-1b44f16d4110\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:50.694528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.694516    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159d30-c714-496e-b901-38bb4ce46d8d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk\" (UID: \"80159d30-c714-496e-b901-38bb4ce46d8d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:50.745724 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.745667    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"]
Oct 30 21:45:50.749771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.749751    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qzlj5"]
Oct 30 21:45:50.795571 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.795541    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr59q\" (UniqueName: \"kubernetes.io/projected/80159d30-c714-496e-b901-38bb4ce46d8d-kube-api-access-rr59q\") pod \"payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk\" (UID: \"80159d30-c714-496e-b901-38bb4ce46d8d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:50.795663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.795621    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3fc218-76e9-41e6-80be-1b44f16d4110-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m\" (UID: \"9c3fc218-76e9-41e6-80be-1b44f16d4110\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:50.795733 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.795718    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7swf\" (UniqueName: \"kubernetes.io/projected/9c3fc218-76e9-41e6-80be-1b44f16d4110-kube-api-access-m7swf\") pod \"payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m\" (UID: \"9c3fc218-76e9-41e6-80be-1b44f16d4110\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:50.795807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.795791    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159d30-c714-496e-b901-38bb4ce46d8d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk\" (UID: \"80159d30-c714-496e-b901-38bb4ce46d8d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:50.796299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.796275    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3fc218-76e9-41e6-80be-1b44f16d4110-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m\" (UID: \"9c3fc218-76e9-41e6-80be-1b44f16d4110\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:50.796582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.796564    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159d30-c714-496e-b901-38bb4ce46d8d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk\" (UID: \"80159d30-c714-496e-b901-38bb4ce46d8d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:50.851078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.851054    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr59q\" (UniqueName: \"kubernetes.io/projected/80159d30-c714-496e-b901-38bb4ce46d8d-kube-api-access-rr59q\") pod \"payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk\" (UID: \"80159d30-c714-496e-b901-38bb4ce46d8d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:50.851178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.851090    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7swf\" (UniqueName: \"kubernetes.io/projected/9c3fc218-76e9-41e6-80be-1b44f16d4110-kube-api-access-m7swf\") pod \"payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m\" (UID: \"9c3fc218-76e9-41e6-80be-1b44f16d4110\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:50.853007 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.852859    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:50.857317 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.857299    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"]
Oct 30 21:45:50.860358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:50.860337    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-v64g9"]
Oct 30 21:45:51.050240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.050164    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"]
Oct 30 21:45:51.145568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.145534    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:51.153756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.153731    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"]
Oct 30 21:45:51.233245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.233196    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"]
Oct 30 21:45:51.239514 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:45:51.239477    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3fc218_76e9_41e6_80be_1b44f16d4110.slice/crio-fceb091a6cb16803b094e44c8c2eac35a76e0604ff712e49174b75a2f57481df WatchSource:0}: Error finding container fceb091a6cb16803b094e44c8c2eac35a76e0604ff712e49174b75a2f57481df: Status 404 returned error can't find the container with id fceb091a6cb16803b094e44c8c2eac35a76e0604ff712e49174b75a2f57481df
Oct 30 21:45:51.536429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.536402    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"]
Oct 30 21:45:51.537546 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:45:51.537518    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80159d30_c714_496e_b901_38bb4ce46d8d.slice/crio-ae81c43c367add235100651094f2cd215b06937cf33ba90802c77beb81fe82ba WatchSource:0}: Error finding container ae81c43c367add235100651094f2cd215b06937cf33ba90802c77beb81fe82ba: Status 404 returned error can't find the container with id ae81c43c367add235100651094f2cd215b06937cf33ba90802c77beb81fe82ba
Oct 30 21:45:51.650574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.650541    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk" event={"ID":"80159d30-c714-496e-b901-38bb4ce46d8d","Type":"ContainerStarted","Data":"ae81c43c367add235100651094f2cd215b06937cf33ba90802c77beb81fe82ba"}
Oct 30 21:45:51.652162 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.652136    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m" event={"ID":"9c3fc218-76e9-41e6-80be-1b44f16d4110","Type":"ContainerStarted","Data":"4e57f21793ea8b70518962f6e8cc62039c5a9c9a30f4ebc7d3ea4f967cb36ff9"}
Oct 30 21:45:51.652309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.652185    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m" event={"ID":"9c3fc218-76e9-41e6-80be-1b44f16d4110","Type":"ContainerStarted","Data":"fceb091a6cb16803b094e44c8c2eac35a76e0604ff712e49174b75a2f57481df"}
Oct 30 21:45:51.652394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.652380    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:45:51.767487 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:51.767445    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m" podStartSLOduration=1.7674310530000001 podStartE2EDuration="1.767431053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:51.766267811 +0000 UTC m=+2007.637386930" watchObservedRunningTime="2026-10-30 21:45:51.767431053 +0000 UTC m=+2007.638550172"
Oct 30 21:45:52.355555 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:52.355521    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:45:52.361275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:52.361252    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03fdf533-e9a8-4410-910b-dfff5aaa73fb" path="/var/lib/kubelet/pods/03fdf533-e9a8-4410-910b-dfff5aaa73fb/volumes"
Oct 30 21:45:52.361727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:52.361714    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6567df37-bf65-475c-bc04-d7f9c68667ad" path="/var/lib/kubelet/pods/6567df37-bf65-475c-bc04-d7f9c68667ad/volumes"
Oct 30 21:45:52.662633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:52.662535    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk" event={"ID":"80159d30-c714-496e-b901-38bb4ce46d8d","Type":"ContainerStarted","Data":"e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee"}
Oct 30 21:45:52.779340 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:52.779276    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk" podStartSLOduration=2.7792577019999998 podStartE2EDuration="2.779257702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:52.777327376 +0000 UTC m=+2008.648446496" watchObservedRunningTime="2026-10-30 21:45:52.779257702 +0000 UTC m=+2008.650376819"
Oct 30 21:45:53.486394 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.486364    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"]
Oct 30 21:45:53.486582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.486559    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6" podUID="4bc20589-8be4-4048-a8ea-05d5a4b325ac" containerName="maas-api" containerID="cri-o://4de71047b91848e101d119428db6f0cf8bf8179b7d25f47e6857d7a036cb1ac5" gracePeriod=30
Oct 30 21:45:53.591426 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.591392    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"]
Oct 30 21:45:53.681477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.681155    5786 generic.go:356] "Generic (PLEG): container finished" podID="4bc20589-8be4-4048-a8ea-05d5a4b325ac" containerID="4de71047b91848e101d119428db6f0cf8bf8179b7d25f47e6857d7a036cb1ac5" exitCode=0
Oct 30 21:45:53.681914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.681848    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6" event={"ID":"4bc20589-8be4-4048-a8ea-05d5a4b325ac","Type":"ContainerDied","Data":"4de71047b91848e101d119428db6f0cf8bf8179b7d25f47e6857d7a036cb1ac5"}
Oct 30 21:45:53.682001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.681924    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:45:53.682001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.681965    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"]
Oct 30 21:45:53.760816 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.760790    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:45:53.773819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.773789    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-maas-api-tls\") pod \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") "
Oct 30 21:45:53.773943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.773929    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bc20589-8be4-4048-a8ea-05d5a4b325ac-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc20589-8be4-4048-a8ea-05d5a4b325ac-openshift-service-ca\") pod \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") "
Oct 30 21:45:53.774034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.774018    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-metrics-certs\") pod \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") "
Oct 30 21:45:53.774154 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.774133    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bc20589-8be4-4048-a8ea-05d5a4b325ac-kube-api-access-5b8m4\" (UniqueName: \"kubernetes.io/projected/4bc20589-8be4-4048-a8ea-05d5a4b325ac-kube-api-access-5b8m4\") pod \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\" (UID: \"4bc20589-8be4-4048-a8ea-05d5a4b325ac\") "
Oct 30 21:45:53.774311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.774267    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc20589-8be4-4048-a8ea-05d5a4b325ac-openshift-service-ca" pod "4bc20589-8be4-4048-a8ea-05d5a4b325ac" (UID: "4bc20589-8be4-4048-a8ea-05d5a4b325ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:53.774810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.774788    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc20589-8be4-4048-a8ea-05d5a4b325ac-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:53.776032 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.776012    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-maas-api-tls" pod "4bc20589-8be4-4048-a8ea-05d5a4b325ac" (UID: "4bc20589-8be4-4048-a8ea-05d5a4b325ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:45:53.776183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.776162    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-metrics-certs" pod "4bc20589-8be4-4048-a8ea-05d5a4b325ac" (UID: "4bc20589-8be4-4048-a8ea-05d5a4b325ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:45:53.776311 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.776284    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc20589-8be4-4048-a8ea-05d5a4b325ac-kube-api-access-5b8m4" pod "4bc20589-8be4-4048-a8ea-05d5a4b325ac" (UID: "4bc20589-8be4-4048-a8ea-05d5a4b325ac"). InnerVolumeSpecName "kube-api-access-5b8m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:53.876476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.876451    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:53.876476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.876476    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b8m4\" (UniqueName: \"kubernetes.io/projected/4bc20589-8be4-4048-a8ea-05d5a4b325ac-kube-api-access-5b8m4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:53.876650 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:53.876485    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc20589-8be4-4048-a8ea-05d5a4b325ac-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:54.702719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:54.702683    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6" event={"ID":"4bc20589-8be4-4048-a8ea-05d5a4b325ac","Type":"ContainerDied","Data":"6969126c4ddac41e14f1322fe2b3c0ffdf297e9c2177ffbc4e8ce1eea4259246"}
Oct 30 21:45:54.702719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:54.702707    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"
Oct 30 21:45:54.703179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:54.702747    5786 scope.go:122] "RemoveContainer" containerID="4de71047b91848e101d119428db6f0cf8bf8179b7d25f47e6857d7a036cb1ac5"
Oct 30 21:45:54.866131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:54.866103    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"]
Oct 30 21:45:54.869197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:54.869178    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e0f967-85b8bdf985-nthc6"]
Oct 30 21:45:56.050560 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.050525    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb" podUID="64b39f91-0891-47c8-bbff-6659f3bc6998" containerName="payload-processing" containerID="cri-o://cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f" gracePeriod=25
Oct 30 21:45:56.154443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.154401    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s" podUID="7e08d338-b1fa-49e1-95e6-b816053cddf0" containerName="payload-pre-processing" containerID="cri-o://5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab" gracePeriod=25
Oct 30 21:45:56.332130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.332102    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:45:56.370059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.370020    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bc20589-8be4-4048-a8ea-05d5a4b325ac" path="/var/lib/kubelet/pods/4bc20589-8be4-4048-a8ea-05d5a4b325ac/volumes"
Oct 30 21:45:56.424244 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.424222    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64b39f91-0891-47c8-bbff-6659f3bc6998-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b39f91-0891-47c8-bbff-6659f3bc6998-plugins-config-volume\") pod \"64b39f91-0891-47c8-bbff-6659f3bc6998\" (UID: \"64b39f91-0891-47c8-bbff-6659f3bc6998\") "
Oct 30 21:45:56.424393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.424381    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64b39f91-0891-47c8-bbff-6659f3bc6998-kube-api-access-vz8nt\" (UniqueName: \"kubernetes.io/projected/64b39f91-0891-47c8-bbff-6659f3bc6998-kube-api-access-vz8nt\") pod \"64b39f91-0891-47c8-bbff-6659f3bc6998\" (UID: \"64b39f91-0891-47c8-bbff-6659f3bc6998\") "
Oct 30 21:45:56.424509 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.424489    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64b39f91-0891-47c8-bbff-6659f3bc6998-plugins-config-volume" pod "64b39f91-0891-47c8-bbff-6659f3bc6998" (UID: "64b39f91-0891-47c8-bbff-6659f3bc6998"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:56.424899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.424877    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b39f91-0891-47c8-bbff-6659f3bc6998-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:56.426595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.426575    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64b39f91-0891-47c8-bbff-6659f3bc6998-kube-api-access-vz8nt" pod "64b39f91-0891-47c8-bbff-6659f3bc6998" (UID: "64b39f91-0891-47c8-bbff-6659f3bc6998"). InnerVolumeSpecName "kube-api-access-vz8nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:56.429073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.429058    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:45:56.526112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.526084    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e08d338-b1fa-49e1-95e6-b816053cddf0-kube-api-access-h2hlt\" (UniqueName: \"kubernetes.io/projected/7e08d338-b1fa-49e1-95e6-b816053cddf0-kube-api-access-h2hlt\") pod \"7e08d338-b1fa-49e1-95e6-b816053cddf0\" (UID: \"7e08d338-b1fa-49e1-95e6-b816053cddf0\") "
Oct 30 21:45:56.526264 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.526117    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e08d338-b1fa-49e1-95e6-b816053cddf0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e08d338-b1fa-49e1-95e6-b816053cddf0-plugins-config-volume\") pod \"7e08d338-b1fa-49e1-95e6-b816053cddf0\" (UID: \"7e08d338-b1fa-49e1-95e6-b816053cddf0\") "
Oct 30 21:45:56.526500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.526477    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz8nt\" (UniqueName: \"kubernetes.io/projected/64b39f91-0891-47c8-bbff-6659f3bc6998-kube-api-access-vz8nt\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:56.526571 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.526506    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e08d338-b1fa-49e1-95e6-b816053cddf0-plugins-config-volume" pod "7e08d338-b1fa-49e1-95e6-b816053cddf0" (UID: "7e08d338-b1fa-49e1-95e6-b816053cddf0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:56.528269 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.528251    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e08d338-b1fa-49e1-95e6-b816053cddf0-kube-api-access-h2hlt" pod "7e08d338-b1fa-49e1-95e6-b816053cddf0" (UID: "7e08d338-b1fa-49e1-95e6-b816053cddf0"). InnerVolumeSpecName "kube-api-access-h2hlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:56.628272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.628183    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2hlt\" (UniqueName: \"kubernetes.io/projected/7e08d338-b1fa-49e1-95e6-b816053cddf0-kube-api-access-h2hlt\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:56.628272 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.628233    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e08d338-b1fa-49e1-95e6-b816053cddf0-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:56.720961 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.720933    5786 generic.go:356] "Generic (PLEG): container finished" podID="64b39f91-0891-47c8-bbff-6659f3bc6998" containerID="cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f" exitCode=0
Oct 30 21:45:56.721093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.720990    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"
Oct 30 21:45:56.721093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.721014    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb" event={"ID":"64b39f91-0891-47c8-bbff-6659f3bc6998","Type":"ContainerDied","Data":"cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f"}
Oct 30 21:45:56.721093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.721065    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb" event={"ID":"64b39f91-0891-47c8-bbff-6659f3bc6998","Type":"ContainerDied","Data":"16953dac0532964431806b17ecdc2203e1558a984781a747e034bcc1ea747611"}
Oct 30 21:45:56.721093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.721090    5786 scope.go:122] "RemoveContainer" containerID="cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f"
Oct 30 21:45:56.722699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.722676    5786 generic.go:356] "Generic (PLEG): container finished" podID="7e08d338-b1fa-49e1-95e6-b816053cddf0" containerID="5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab" exitCode=0
Oct 30 21:45:56.722794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.722709    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"
Oct 30 21:45:56.722837 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.722815    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s" event={"ID":"7e08d338-b1fa-49e1-95e6-b816053cddf0","Type":"ContainerDied","Data":"5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab"}
Oct 30 21:45:56.722875 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.722865    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s" event={"ID":"7e08d338-b1fa-49e1-95e6-b816053cddf0","Type":"ContainerDied","Data":"b90b563a14931ed4c17e1a2894b14c5b6ba4c30933e4bf5c87c847e1621a2f39"}
Oct 30 21:45:56.732948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.732932    5786 scope.go:122] "RemoveContainer" containerID="cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f"
Oct 30 21:45:56.733264 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:56.733245    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f\": container with ID starting with cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f not found: ID does not exist" containerID="cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f"
Oct 30 21:45:56.733349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.733270    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f"} err="failed to get container status \"cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f\": rpc error: code = NotFound desc = could not find container \"cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f\": container with ID starting with cc27831dec091866b55841c1b9fcf44b4f534e4f392eb7b114ada2d5db82588f not found: ID does not exist"
Oct 30 21:45:56.733349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.733295    5786 scope.go:122] "RemoveContainer" containerID="5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab"
Oct 30 21:45:56.742387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.742371    5786 scope.go:122] "RemoveContainer" containerID="5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab"
Oct 30 21:45:56.742640 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:56.742622    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab\": container with ID starting with 5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab not found: ID does not exist" containerID="5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab"
Oct 30 21:45:56.742706 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.742648    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab"} err="failed to get container status \"5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab\": rpc error: code = NotFound desc = could not find container \"5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab\": container with ID starting with 5745b2bbf0ae6c92e85057f9718a4b458c9d31b5868a0ca223afb18959a024ab not found: ID does not exist"
Oct 30 21:45:56.884234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.884158    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"]
Oct 30 21:45:56.887341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.887319    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-815dab-7b9cddf46c-cbw8s"]
Oct 30 21:45:56.993622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.993589    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"]
Oct 30 21:45:56.996925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:56.996903    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-815dab-6cc5bd7b67-t8nnb"]
Oct 30 21:45:57.302235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:57.284084    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"]
Oct 30 21:45:57.302235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:57.284340    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" containerID="cri-o://5a046ac40b95bf39d444cec9b753e4a4a1bdbf3b9b85bd1c61c50126f598bf1b" gracePeriod=30
Oct 30 21:45:57.302235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:57.289459    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"]
Oct 30 21:45:57.302235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:57.289664    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy" containerID="cri-o://c9f73d7c1bf21c7c122ce97f6434a36473828be0c9f373be01f863b66f6ada85" gracePeriod=30
Oct 30 21:45:57.302235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:57.293487    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"]
Oct 30 21:45:57.302235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:57.293686    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" containerID="cri-o://ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f" gracePeriod=30
Oct 30 21:45:58.360876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:58.360842    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64b39f91-0891-47c8-bbff-6659f3bc6998" path="/var/lib/kubelet/pods/64b39f91-0891-47c8-bbff-6659f3bc6998/volumes"
Oct 30 21:45:58.361255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:58.361244    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e08d338-b1fa-49e1-95e6-b816053cddf0" path="/var/lib/kubelet/pods/7e08d338-b1fa-49e1-95e6-b816053cddf0/volumes"
Oct 30 21:45:58.592509 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:58.592475    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp" podUID="a4a35a96-a63c-4294-88e8-0203a394417c" containerName="payload-processing" containerID="cri-o://a434860fc45602662630c8b6e85451c2a599c79312ce0c02d950d81f6508b5f2" gracePeriod=25
Oct 30 21:45:58.682412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:58.682385    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z" podUID="308f38d8-cf2d-4dff-ab54-a2c36e36f3e8" containerName="payload-pre-processing" containerID="cri-o://1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123" gracePeriod=25
Oct 30 21:45:58.740135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:58.740115    5786 generic.go:356] "Generic (PLEG): container finished" podID="a4a35a96-a63c-4294-88e8-0203a394417c" containerID="a434860fc45602662630c8b6e85451c2a599c79312ce0c02d950d81f6508b5f2" exitCode=0
Oct 30 21:45:58.740257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:58.740161    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp" event={"ID":"a4a35a96-a63c-4294-88e8-0203a394417c","Type":"ContainerDied","Data":"a434860fc45602662630c8b6e85451c2a599c79312ce0c02d950d81f6508b5f2"}
Oct 30 21:45:58.962572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:58.962549    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:45:59.018485 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.018451    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:45:59.093989 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.093962    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-kube-api-access-25g5b\" (UniqueName: \"kubernetes.io/projected/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-kube-api-access-25g5b\") pod \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\" (UID: \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\") "
Oct 30 21:45:59.094147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.094026    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4a35a96-a63c-4294-88e8-0203a394417c-kube-api-access-qlmvk\" (UniqueName: \"kubernetes.io/projected/a4a35a96-a63c-4294-88e8-0203a394417c-kube-api-access-qlmvk\") pod \"a4a35a96-a63c-4294-88e8-0203a394417c\" (UID: \"a4a35a96-a63c-4294-88e8-0203a394417c\") "
Oct 30 21:45:59.094147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.094068    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-plugins-config-volume\") pod \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\" (UID: \"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8\") "
Oct 30 21:45:59.094314 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.094267    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4a35a96-a63c-4294-88e8-0203a394417c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a35a96-a63c-4294-88e8-0203a394417c-plugins-config-volume\") pod \"a4a35a96-a63c-4294-88e8-0203a394417c\" (UID: \"a4a35a96-a63c-4294-88e8-0203a394417c\") "
Oct 30 21:45:59.095051 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.095025    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a35a96-a63c-4294-88e8-0203a394417c-plugins-config-volume" pod "a4a35a96-a63c-4294-88e8-0203a394417c" (UID: "a4a35a96-a63c-4294-88e8-0203a394417c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:59.095332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.095315    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-plugins-config-volume" pod "308f38d8-cf2d-4dff-ab54-a2c36e36f3e8" (UID: "308f38d8-cf2d-4dff-ab54-a2c36e36f3e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:59.097072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.097046    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a35a96-a63c-4294-88e8-0203a394417c-kube-api-access-qlmvk" pod "a4a35a96-a63c-4294-88e8-0203a394417c" (UID: "a4a35a96-a63c-4294-88e8-0203a394417c"). InnerVolumeSpecName "kube-api-access-qlmvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:59.098117 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.098089    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-kube-api-access-25g5b" pod "308f38d8-cf2d-4dff-ab54-a2c36e36f3e8" (UID: "308f38d8-cf2d-4dff-ab54-a2c36e36f3e8"). InnerVolumeSpecName "kube-api-access-25g5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:59.196036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.196013    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25g5b\" (UniqueName: \"kubernetes.io/projected/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-kube-api-access-25g5b\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:59.196036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.196038    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlmvk\" (UniqueName: \"kubernetes.io/projected/a4a35a96-a63c-4294-88e8-0203a394417c-kube-api-access-qlmvk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:59.196191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.196047    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:59.196191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.196056    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a35a96-a63c-4294-88e8-0203a394417c-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:59.552552 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.552473    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"]
Oct 30 21:45:59.553332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553312    5786 cpu_manager.go:414] "containerMap: removing container" podUID="308f38d8-cf2d-4dff-ab54-a2c36e36f3e8" containerName="payload-pre-processing"
Oct 30 21:45:59.553436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553333    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="308f38d8-cf2d-4dff-ab54-a2c36e36f3e8" containerName="payload-pre-processing"
Oct 30 21:45:59.553436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553351    5786 cpu_manager.go:414] "containerMap: removing container" podUID="4bc20589-8be4-4048-a8ea-05d5a4b325ac" containerName="maas-api"
Oct 30 21:45:59.553436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553358    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bc20589-8be4-4048-a8ea-05d5a4b325ac" containerName="maas-api"
Oct 30 21:45:59.553436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553426    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a4a35a96-a63c-4294-88e8-0203a394417c" containerName="payload-processing"
Oct 30 21:45:59.553436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553434    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4a35a96-a63c-4294-88e8-0203a394417c" containerName="payload-processing"
Oct 30 21:45:59.553710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553446    5786 cpu_manager.go:414] "containerMap: removing container" podUID="64b39f91-0891-47c8-bbff-6659f3bc6998" containerName="payload-processing"
Oct 30 21:45:59.553710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553453    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64b39f91-0891-47c8-bbff-6659f3bc6998" containerName="payload-processing"
Oct 30 21:45:59.553710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553477    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7e08d338-b1fa-49e1-95e6-b816053cddf0" containerName="payload-pre-processing"
Oct 30 21:45:59.553710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553487    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e08d338-b1fa-49e1-95e6-b816053cddf0" containerName="payload-pre-processing"
Oct 30 21:45:59.553710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553663    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e08d338-b1fa-49e1-95e6-b816053cddf0" containerName="payload-pre-processing"
Oct 30 21:45:59.553710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553684    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="308f38d8-cf2d-4dff-ab54-a2c36e36f3e8" containerName="payload-pre-processing"
Oct 30 21:45:59.553710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553696    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bc20589-8be4-4048-a8ea-05d5a4b325ac" containerName="maas-api"
Oct 30 21:45:59.553710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553711    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4a35a96-a63c-4294-88e8-0203a394417c" containerName="payload-processing"
Oct 30 21:45:59.554067 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.553722    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="64b39f91-0891-47c8-bbff-6659f3bc6998" containerName="payload-processing"
Oct 30 21:45:59.558552 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.558531    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.568303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.568274    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"]
Oct 30 21:45:59.577438 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.577409    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.626253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.623787    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"]
Oct 30 21:45:59.688575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.688549    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"]
Oct 30 21:45:59.703282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703250    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.703409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703307    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.703468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703426    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.703468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703458    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.703558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703475    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.703558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703492    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.703558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703549    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.703663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703624    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.703663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703644    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.703762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703683    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.703819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703783    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv92b\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-kube-api-access-hv92b\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.703902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703885    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.703964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703932    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.704017 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.703971    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snbpw\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-kube-api-access-snbpw\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.704017 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.704002    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.704088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.704053    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.704088 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.704078    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.704160 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.704118    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.750536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.750511    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"
Oct 30 21:45:59.750536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.750521    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp" event={"ID":"a4a35a96-a63c-4294-88e8-0203a394417c","Type":"ContainerDied","Data":"a568cd38ea107093a195a2629ea5d495fb8e531ded59830b351d7b6b2d69ee61"}
Oct 30 21:45:59.750776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.750589    5786 scope.go:122] "RemoveContainer" containerID="a434860fc45602662630c8b6e85451c2a599c79312ce0c02d950d81f6508b5f2"
Oct 30 21:45:59.752547 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.752521    5786 generic.go:356] "Generic (PLEG): container finished" podID="308f38d8-cf2d-4dff-ab54-a2c36e36f3e8" containerID="1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123" exitCode=0
Oct 30 21:45:59.752653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.752639    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z" event={"ID":"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8","Type":"ContainerDied","Data":"1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123"}
Oct 30 21:45:59.752691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.752682    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z" event={"ID":"308f38d8-cf2d-4dff-ab54-a2c36e36f3e8","Type":"ContainerDied","Data":"aa227eeb9036d5f690464a5f6283640021a4c85f61d25b79949ec59b41f05c95"}
Oct 30 21:45:59.752771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.752760    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"
Oct 30 21:45:59.761632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.761535    5786 scope.go:122] "RemoveContainer" containerID="1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123"
Oct 30 21:45:59.770736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.770718    5786 scope.go:122] "RemoveContainer" containerID="1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123"
Oct 30 21:45:59.770969 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:45:59.770948    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123\": container with ID starting with 1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123 not found: ID does not exist" containerID="1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123"
Oct 30 21:45:59.771014 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.770976    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123"} err="failed to get container status \"1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123\": rpc error: code = NotFound desc = could not find container \"1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123\": container with ID starting with 1110c74ab1caca4d1c240f9ea2ceee3154d97eb050ad71ae30646344e338f123 not found: ID does not exist"
Oct 30 21:45:59.805862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.805815    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snbpw\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-kube-api-access-snbpw\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.805862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.805846    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.805984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.805892    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.805984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.805913    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.805984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.805957    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.806161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.805997    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.806161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806044    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806161 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806124    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.806333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806164    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806190    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.806333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806237    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.806333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806278    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806305    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806330    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806368    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.806577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806379    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806449    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806369    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.806577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806517    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv92b\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-kube-api-access-hv92b\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806575    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806594    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.806774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806636    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806774 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806696    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.806959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806939    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.807021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.806963    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.807139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.807117    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.807263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.807149    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.807263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.807164    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.808981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.808962    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.809099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.809082    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.809310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.809294    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.809594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.809576    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.861929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.861907    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.862066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.861977    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snbpw\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-kube-api-access-snbpw\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.862131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.862070    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv92b\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-kube-api-access-hv92b\") pod \"maas-default-gateway-openshift-default-6cc458d887-nsg54\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:45:59.862131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.862113    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-cp7fj\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.901467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.901441    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"]
Oct 30 21:45:59.904504 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.904483    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:45:59.904624 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.904606    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e0f967-674bdd5947-vqvcp"]
Oct 30 21:45:59.928574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:45:59.928551    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:46:00.010725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.010700    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"]
Oct 30 21:46:00.015644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.015619    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e0f967-54575d79ff-59p2z"]
Oct 30 21:46:00.341655 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.341627    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"]
Oct 30 21:46:00.345891 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:46:00.345677    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda988c4e4_539d_4c8d_bc65_bd3298b39ee6.slice/crio-f1dca00d79b587d529cab8f177ce02b13ce3673ff38b4fbb490c86666d1bfd66 WatchSource:0}: Error finding container f1dca00d79b587d529cab8f177ce02b13ce3673ff38b4fbb490c86666d1bfd66: Status 404 returned error can't find the container with id f1dca00d79b587d529cab8f177ce02b13ce3673ff38b4fbb490c86666d1bfd66
Oct 30 21:46:00.346194 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:46:00.345987    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa95468_9ab2_4c12_82b9_a9a8793ee3ed.slice/crio-d3cbcaad12fe1d6d5b0b401373c2bb7dbf0615f45370d1519c4d837eb0acd9ad WatchSource:0}: Error finding container d3cbcaad12fe1d6d5b0b401373c2bb7dbf0615f45370d1519c4d837eb0acd9ad: Status 404 returned error can't find the container with id d3cbcaad12fe1d6d5b0b401373c2bb7dbf0615f45370d1519c4d837eb0acd9ad
Oct 30 21:46:00.346346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.346323    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"]
Oct 30 21:46:00.348850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.348052    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:46:00.348850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.348126    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:46:00.348850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.348163    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:46:00.348850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.348588    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:46:00.348850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.348643    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:46:00.348850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.348700    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:46:00.359599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.359579    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="308f38d8-cf2d-4dff-ab54-a2c36e36f3e8" path="/var/lib/kubelet/pods/308f38d8-cf2d-4dff-ab54-a2c36e36f3e8/volumes"
Oct 30 21:46:00.359968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.359955    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4a35a96-a63c-4294-88e8-0203a394417c" path="/var/lib/kubelet/pods/a4a35a96-a63c-4294-88e8-0203a394417c/volumes"
Oct 30 21:46:00.750033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.749998    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:46:00.750486 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.750048    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:46:00.765810 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.765780    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" event={"ID":"caa95468-9ab2-4c12-82b9-a9a8793ee3ed","Type":"ContainerStarted","Data":"65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b"}
Oct 30 21:46:00.765994 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.765831    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" event={"ID":"caa95468-9ab2-4c12-82b9-a9a8793ee3ed","Type":"ContainerStarted","Data":"d3cbcaad12fe1d6d5b0b401373c2bb7dbf0615f45370d1519c4d837eb0acd9ad"}
Oct 30 21:46:00.768769 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.768744    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" event={"ID":"a988c4e4-539d-4c8d-bc65-bd3298b39ee6","Type":"ContainerStarted","Data":"be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84"}
Oct 30 21:46:00.768863 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.768793    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" event={"ID":"a988c4e4-539d-4c8d-bc65-bd3298b39ee6","Type":"ContainerStarted","Data":"f1dca00d79b587d529cab8f177ce02b13ce3673ff38b4fbb490c86666d1bfd66"}
Oct 30 21:46:00.890326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.890275    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podStartSLOduration=1.8902611660000002 podStartE2EDuration="1.890261166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:00.888710215 +0000 UTC m=+2016.759829333" watchObservedRunningTime="2026-10-30 21:46:00.890261166 +0000 UTC m=+2016.761380284"
Oct 30 21:46:00.905137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.905112    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:46:00.929554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.929535    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:46:00.988651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.988618    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"]
Oct 30 21:46:00.991931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:00.991904    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e0f967-mk9j7"]
Oct 30 21:46:01.016281 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:01.014426    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podStartSLOduration=2.014405868 podStartE2EDuration="2.014405868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:01.004126744 +0000 UTC m=+2016.875245863" watchObservedRunningTime="2026-10-30 21:46:01.014405868 +0000 UTC m=+2016.885524987"
Oct 30 21:46:01.907240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:01.907189    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 21:46:01.907854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:01.907257    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 21:46:01.930115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:01.930086    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused" start-of-body=
Oct 30 21:46:01.930265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:01.930131    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused"
Oct 30 21:46:02.414381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.414342    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6597ed43-c6f1-4b6b-b111-24557a9db736" path="/var/lib/kubelet/pods/6597ed43-c6f1-4b6b-b111-24557a9db736/volumes"
Oct 30 21:46:02.768927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.768359    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:46:02.768927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.768418    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 21:46:02.824255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.823934    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:46:02.843577 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.843550    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:46:02.844144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.843990    5786 generic.go:356] "Generic (PLEG): container finished" podID="d314cad3-3324-480b-81f6-9e102076a008" containerID="c9f73d7c1bf21c7c122ce97f6434a36473828be0c9f373be01f863b66f6ada85" exitCode=0
Oct 30 21:46:02.844144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.844092    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" event={"ID":"d314cad3-3324-480b-81f6-9e102076a008","Type":"ContainerDied","Data":"c9f73d7c1bf21c7c122ce97f6434a36473828be0c9f373be01f863b66f6ada85"}
Oct 30 21:46:02.846071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.845883    5786 generic.go:356] "Generic (PLEG): container finished" podID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerID="5a046ac40b95bf39d444cec9b753e4a4a1bdbf3b9b85bd1c61c50126f598bf1b" exitCode=0
Oct 30 21:46:02.846071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.845959    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" event={"ID":"e9712d6e-b3fd-4be6-8711-e401a193579d","Type":"ContainerDied","Data":"5a046ac40b95bf39d444cec9b753e4a4a1bdbf3b9b85bd1c61c50126f598bf1b"}
Oct 30 21:46:02.850345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.848802    5786 generic.go:356] "Generic (PLEG): container finished" podID="ca508309-2ab8-420f-a9d1-f26963632e25" containerID="ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f" exitCode=0
Oct 30 21:46:02.850345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.849564    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"
Oct 30 21:46:02.850345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.849661    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" event={"ID":"ca508309-2ab8-420f-a9d1-f26963632e25","Type":"ContainerDied","Data":"ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f"}
Oct 30 21:46:02.850345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.849686    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh" event={"ID":"ca508309-2ab8-420f-a9d1-f26963632e25","Type":"ContainerDied","Data":"364eedf102698b2193bb41e4989495beaf53fb3da768d644a991ac1cbf43cb57"}
Oct 30 21:46:02.850345 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.849711    5786 scope.go:122] "RemoveContainer" containerID="ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f"
Oct 30 21:46:02.906353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.905892    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 21:46:02.906353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.905960    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 21:46:02.918711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.918689    5786 scope.go:122] "RemoveContainer" containerID="3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3"
Oct 30 21:46:02.929698 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.929670    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused" start-of-body=
Oct 30 21:46:02.929797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.929723    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused"
Oct 30 21:46:02.932949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.931532    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:46:02.932949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.932781    5786 scope.go:122] "RemoveContainer" containerID="ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f"
Oct 30 21:46:02.933193 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:46:02.933171    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f\": container with ID starting with ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f not found: ID does not exist" containerID="ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f"
Oct 30 21:46:02.933279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.933223    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f"} err="failed to get container status \"ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f\": rpc error: code = NotFound desc = could not find container \"ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f\": container with ID starting with ba355d7b29418c5574e6c5dd367c3ff587d94f910a5596db7ec3b3eab1c8256f not found: ID does not exist"
Oct 30 21:46:02.933700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.933681    5786 scope.go:122] "RemoveContainer" containerID="3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3"
Oct 30 21:46:02.934032 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:46:02.934008    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3\": container with ID starting with 3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3 not found: ID does not exist" containerID="3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3"
Oct 30 21:46:02.934070 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.934043    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3"} err="failed to get container status \"3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3\": rpc error: code = NotFound desc = could not find container \"3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3\": container with ID starting with 3f8666b4f8280fe64e42a5f0a9f22e63b0aaf319130440073b72c32795f1d9a3 not found: ID does not exist"
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.979620    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-kube-api-access-s4jkm\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-kube-api-access-s4jkm\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.979702    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-istio-token\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-istio-token\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.979751    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-envoy\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.979919    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-certs\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.979966    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca508309-2ab8-420f-a9d1-f26963632e25-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca508309-2ab8-420f-a9d1-f26963632e25-istio-podinfo\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.979995    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-socket\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.980066    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-data\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.980094    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-credential-socket\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:02.997286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:02.980244    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca508309-2ab8-420f-a9d1-f26963632e25-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca508309-2ab8-420f-a9d1-f26963632e25-istiod-ca-cert\") pod \"ca508309-2ab8-420f-a9d1-f26963632e25\" (UID: \"ca508309-2ab8-420f-a9d1-f26963632e25\") "
Oct 30 21:46:03.011400 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.011281    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-socket" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.014493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.014420    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca508309-2ab8-420f-a9d1-f26963632e25-istio-podinfo" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:46:03.014660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.014636    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-data" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.014752 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.014723    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca508309-2ab8-420f-a9d1-f26963632e25-istiod-ca-cert" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:03.014815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.014801    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-credential-socket" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.014871 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.014853    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-certs" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.019231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.017569    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-kube-api-access-s4jkm" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "kube-api-access-s4jkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:03.019231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.018469    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-envoy" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.019399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.019337    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-istio-token" pod "ca508309-2ab8-420f-a9d1-f26963632e25" (UID: "ca508309-2ab8-420f-a9d1-f26963632e25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:03.117692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.117654    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-envoy\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.117932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.117911    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-certs\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.118100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.118084    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-data\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.118288 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.118270    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-credential-socket\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.118472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.118453    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d314cad3-3324-480b-81f6-9e102076a008-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d314cad3-3324-480b-81f6-9e102076a008-istio-podinfo\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.118609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.118326    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-certs" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.118687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.118570    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-data" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.118766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.118596    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-istio-token\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-istio-token\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.118897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.118851    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-credential-socket" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.118970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.118954    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d314cad3-3324-480b-81f6-9e102076a008-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d314cad3-3324-480b-81f6-9e102076a008-istiod-ca-cert\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119036    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-kube-api-access-pc8kf\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-kube-api-access-pc8kf\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119103    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-socket\") pod \"d314cad3-3324-480b-81f6-9e102076a008\" (UID: \"d314cad3-3324-480b-81f6-9e102076a008\") "
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119367    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d314cad3-3324-480b-81f6-9e102076a008-istiod-ca-cert" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119565    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-socket" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119881    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119903    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d314cad3-3324-480b-81f6-9e102076a008-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119918    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca508309-2ab8-420f-a9d1-f26963632e25-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119932    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119944    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119958    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119971    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119984    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca508309-2ab8-420f-a9d1-f26963632e25-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.119997    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.120010    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.120022    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4jkm\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-kube-api-access-s4jkm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.120037    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.120052    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca508309-2ab8-420f-a9d1-f26963632e25-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.120271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.120069    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca508309-2ab8-420f-a9d1-f26963632e25-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.123829 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.123084    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-kube-api-access-pc8kf" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "kube-api-access-pc8kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:03.123829 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.123122    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-istio-token" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:03.125935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.125886    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d314cad3-3324-480b-81f6-9e102076a008-istio-podinfo" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:46:03.128776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.128079    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-envoy" pod "d314cad3-3324-480b-81f6-9e102076a008" (UID: "d314cad3-3324-480b-81f6-9e102076a008"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.221143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.221112    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc8kf\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-kube-api-access-pc8kf\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.221143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.221145    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d314cad3-3324-480b-81f6-9e102076a008-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.221369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.221160    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d314cad3-3324-480b-81f6-9e102076a008-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.221369 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.221173    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d314cad3-3324-480b-81f6-9e102076a008-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.234097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.234070    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:46:03.322845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.322816    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-kube-api-access-gwmls\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-kube-api-access-gwmls\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.322982 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.322911    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-token\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-token\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.323072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.322978    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-certs\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.323072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323004    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-credential-socket\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.323072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323053    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-socket\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.323268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323097    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9712d6e-b3fd-4be6-8711-e401a193579d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9712d6e-b3fd-4be6-8711-e401a193579d-istiod-ca-cert\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.323268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323190    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-data\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.323395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323310    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-envoy\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.323395 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323348    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-podinfo\") pod \"e9712d6e-b3fd-4be6-8711-e401a193579d\" (UID: \"e9712d6e-b3fd-4be6-8711-e401a193579d\") "
Oct 30 21:46:03.323504 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323466    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-socket" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.323688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323664    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-credential-socket" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.323983 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.323960    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-data" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.324262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.324224    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.324262 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.324249    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.324406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.324264    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.324593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.324571    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9712d6e-b3fd-4be6-8711-e401a193579d-istiod-ca-cert" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:03.325198 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.325168    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-certs" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.325891 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.325863    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-token" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:03.325988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.325921    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-kube-api-access-gwmls" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "kube-api-access-gwmls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:03.326337 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.326313    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-envoy" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:03.327462 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.327436    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-podinfo" pod "e9712d6e-b3fd-4be6-8711-e401a193579d" (UID: "e9712d6e-b3fd-4be6-8711-e401a193579d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:46:03.386237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.381260    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"]
Oct 30 21:46:03.407232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.405323    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ddmwh"]
Oct 30 21:46:03.426347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.426246    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.426347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.426273    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9712d6e-b3fd-4be6-8711-e401a193579d-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.426347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.426289    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.426347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.426306    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.426347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.426319    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwmls\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-kube-api-access-gwmls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.426347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.426333    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9712d6e-b3fd-4be6-8711-e401a193579d-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:03.792230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.791296    5786 patch_prober.go:29] interesting pod/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:03.792230 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.791368    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:03.893853 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.893813    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr" event={"ID":"d314cad3-3324-480b-81f6-9e102076a008","Type":"ContainerDied","Data":"34525a25651ebf8645bf15cd4af76abb5bc48f891dfc9d9ea31e8c95914d3cde"}
Oct 30 21:46:03.894134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.894118    5786 scope.go:122] "RemoveContainer" containerID="c9f73d7c1bf21c7c122ce97f6434a36473828be0c9f373be01f863b66f6ada85"
Oct 30 21:46:03.894379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.894363    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"
Oct 30 21:46:03.917357 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.916732    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 21:46:03.917357 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.916786    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 21:46:03.936092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.934569    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused" start-of-body=
Oct 30 21:46:03.936092 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:03.934630    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused"
Oct 30 21:46:04.050692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.048852    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k" event={"ID":"e9712d6e-b3fd-4be6-8711-e401a193579d","Type":"ContainerDied","Data":"5affa8ca1ee90b17bb5dd3943cb0684b165854445615e64d709b7ed2e6f1f64e"}
Oct 30 21:46:04.050692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.048997    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"
Oct 30 21:46:04.132045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.131817    5786 scope.go:122] "RemoveContainer" containerID="5a046ac40b95bf39d444cec9b753e4a4a1bdbf3b9b85bd1c61c50126f598bf1b"
Oct 30 21:46:04.308653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.308577    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"]
Oct 30 21:46:04.312116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.312094    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-ss8mr"]
Oct 30 21:46:04.425623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.425588    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" path="/var/lib/kubelet/pods/ca508309-2ab8-420f-a9d1-f26963632e25/volumes"
Oct 30 21:46:04.426870 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.426844    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d314cad3-3324-480b-81f6-9e102076a008" path="/var/lib/kubelet/pods/d314cad3-3324-480b-81f6-9e102076a008/volumes"
Oct 30 21:46:04.427191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.427177    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"]
Oct 30 21:46:04.427266 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.427225    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a4eab-openshift-default-868987fdc7-hdk5k"]
Oct 30 21:46:04.706177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.706153    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:46:04.910424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.909875    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 21:46:04.910424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.909949    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 21:46:04.929752 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.929719    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused" start-of-body=
Oct 30 21:46:04.929913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:04.929772    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused"
Oct 30 21:46:05.905236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:05.904753    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 21:46:05.905236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:05.904806    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 21:46:05.928876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:05.928848    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused" start-of-body=
Oct 30 21:46:05.929025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:05.928888    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused"
Oct 30 21:46:06.407148 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:06.407116    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" path="/var/lib/kubelet/pods/e9712d6e-b3fd-4be6-8711-e401a193579d/volumes"
Oct 30 21:46:06.905476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:06.905434    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 21:46:06.905857 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:06.905500    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 21:46:06.929775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:06.929744    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused" start-of-body=
Oct 30 21:46:06.929894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:06.929795    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused"
Oct 30 21:46:07.905402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:07.905369    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 21:46:07.905569 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:07.905424    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 21:46:07.929788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:07.929760    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused" start-of-body=
Oct 30 21:46:07.929895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:07.929813    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": dial tcp 10.134.0.200:15021: connect: connection refused"
Oct 30 21:46:09.947364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:09.947327    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:09.947762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:09.947386    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:09.970658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:09.970330    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:09.970658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:09.970379    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:10.580727 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.580691    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"]
Oct 30 21:46:10.589241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.588974    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy"
Oct 30 21:46:10.589241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.589002    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy"
Oct 30 21:46:10.589241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.589036    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy"
Oct 30 21:46:10.589241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.589057    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy"
Oct 30 21:46:10.589241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.589079    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy"
Oct 30 21:46:10.589241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.589088    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy"
Oct 30 21:46:10.589241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.589148    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy"
Oct 30 21:46:10.589241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.589156    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy"
Oct 30 21:46:10.594973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.594580    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy"
Oct 30 21:46:10.594973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.594621    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d314cad3-3324-480b-81f6-9e102076a008" containerName="istio-proxy"
Oct 30 21:46:10.594973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.594642    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca508309-2ab8-420f-a9d1-f26963632e25" containerName="istio-proxy"
Oct 30 21:46:10.594973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.594665    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9712d6e-b3fd-4be6-8711-e401a193579d" containerName="istio-proxy"
Oct 30 21:46:10.616481 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.616458    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.638107 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.638080    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"]
Oct 30 21:46:10.703898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.703874    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:46:10.704970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.704951    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:46:10.720137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.719888    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.720137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.719938    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fchgj\" (UniqueName: \"kubernetes.io/projected/5232f0a0-8b9a-4f57-8358-f09a856714ba-kube-api-access-fchgj\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.720137 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.719959    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.827459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.824933    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.827459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.825031    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fchgj\" (UniqueName: \"kubernetes.io/projected/5232f0a0-8b9a-4f57-8358-f09a856714ba-kube-api-access-fchgj\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.827459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.825063    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.827459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.826031    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.842905 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.842682    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.882157 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.882042    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fchgj\" (UniqueName: \"kubernetes.io/projected/5232f0a0-8b9a-4f57-8358-f09a856714ba-kube-api-access-fchgj\") pod \"maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:10.931474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.931448    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:11.002412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.992284    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-cp7fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:11.002412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:10.992342    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:11.016240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.012311    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-nsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.200:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:11.016240 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.012363    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.200:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:11.371328 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:46:11.371284    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5232f0a0_8b9a_4f57_8358_f09a856714ba.slice/crio-804657e0f4c50e898139b53c42c34be4124375374131a2a769975b7dda6e0630 WatchSource:0}: Error finding container 804657e0f4c50e898139b53c42c34be4124375374131a2a769975b7dda6e0630: Status 404 returned error can't find the container with id 804657e0f4c50e898139b53c42c34be4124375374131a2a769975b7dda6e0630
Oct 30 21:46:11.371776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.371750    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"]
Oct 30 21:46:11.393000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.392963    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:46:11.393454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.393432    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:46:11.403498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.403480    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:46:11.403754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.403731    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:46:11.404498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.404484    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:46:11.456194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:11.456178    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:46:12.194554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:12.194510    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n" event={"ID":"5232f0a0-8b9a-4f57-8358-f09a856714ba","Type":"ContainerStarted","Data":"734ff0083b13d63d58f07055a7c2f50f1767ad519def984209ced8b464f08e40"}
Oct 30 21:46:12.194935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:12.194574    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n" event={"ID":"5232f0a0-8b9a-4f57-8358-f09a856714ba","Type":"ContainerStarted","Data":"804657e0f4c50e898139b53c42c34be4124375374131a2a769975b7dda6e0630"}
Oct 30 21:46:12.309478 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:12.309434    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n" podStartSLOduration=2.30942024 podStartE2EDuration="2.30942024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:12.30702384 +0000 UTC m=+2028.178142958" watchObservedRunningTime="2026-10-30 21:46:12.30942024 +0000 UTC m=+2028.180539358"
Oct 30 21:46:13.203871 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:13.203838    5786 generic.go:356] "Generic (PLEG): container finished" podID="5232f0a0-8b9a-4f57-8358-f09a856714ba" containerID="734ff0083b13d63d58f07055a7c2f50f1767ad519def984209ced8b464f08e40" exitCode=0
Oct 30 21:46:13.204343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:13.203931    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n" event={"ID":"5232f0a0-8b9a-4f57-8358-f09a856714ba","Type":"ContainerDied","Data":"734ff0083b13d63d58f07055a7c2f50f1767ad519def984209ced8b464f08e40"}
Oct 30 21:46:14.372191 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.372166    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:14.474468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.474392    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5232f0a0-8b9a-4f57-8358-f09a856714ba-kube-api-access-fchgj\" (UniqueName: \"kubernetes.io/projected/5232f0a0-8b9a-4f57-8358-f09a856714ba-kube-api-access-fchgj\") pod \"5232f0a0-8b9a-4f57-8358-f09a856714ba\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") "
Oct 30 21:46:14.474606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.474533    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-openshift-ca\") pod \"5232f0a0-8b9a-4f57-8358-f09a856714ba\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") "
Oct 30 21:46:14.474756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.474732    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-xks-ca\") pod \"5232f0a0-8b9a-4f57-8358-f09a856714ba\" (UID: \"5232f0a0-8b9a-4f57-8358-f09a856714ba\") "
Oct 30 21:46:14.475150 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.474940    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-openshift-ca" pod "5232f0a0-8b9a-4f57-8358-f09a856714ba" (UID: "5232f0a0-8b9a-4f57-8358-f09a856714ba"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:14.475767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.475744    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:14.479144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.477349    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5232f0a0-8b9a-4f57-8358-f09a856714ba-kube-api-access-fchgj" pod "5232f0a0-8b9a-4f57-8358-f09a856714ba" (UID: "5232f0a0-8b9a-4f57-8358-f09a856714ba"). InnerVolumeSpecName "kube-api-access-fchgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:14.479144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.477459    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-xks-ca" pod "5232f0a0-8b9a-4f57-8358-f09a856714ba" (UID: "5232f0a0-8b9a-4f57-8358-f09a856714ba"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:46:14.576878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.576853    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5232f0a0-8b9a-4f57-8358-f09a856714ba-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:14.576878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:14.576879    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fchgj\" (UniqueName: \"kubernetes.io/projected/5232f0a0-8b9a-4f57-8358-f09a856714ba-kube-api-access-fchgj\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:15.080108 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:15.080074    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"]
Oct 30 21:46:15.097201 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:15.097173    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"]
Oct 30 21:46:15.227309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:15.227189    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"
Oct 30 21:46:15.227309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:15.227251    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n" event={"ID":"5232f0a0-8b9a-4f57-8358-f09a856714ba","Type":"ContainerDied","Data":"804657e0f4c50e898139b53c42c34be4124375374131a2a769975b7dda6e0630"}
Oct 30 21:46:15.227593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:15.227330    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="804657e0f4c50e898139b53c42c34be4124375374131a2a769975b7dda6e0630"
Oct 30 21:46:22.358033 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:46:22.357999    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:46:22.630362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:22.630280    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"]
Oct 30 21:46:26.798549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:26.798513    5786 scope.go:122] "RemoveContainer" containerID="c85f0ca8291338e10e380eea65fbde8e71e2e8dee733419a3cc0d788547acf87"
Oct 30 21:46:26.809135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:26.809113    5786 scope.go:122] "RemoveContainer" containerID="5565e5f262aad34e1a25ef55655b85483ebe369dbb4e7d8599f557395ad56588"
Oct 30 21:46:27.631523 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:27.631464    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s" podUID="70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" containerName="payload-pre-processing" containerID="cri-o://d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9" gracePeriod=25
Oct 30 21:46:30.101776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.101739    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main" containerID="cri-o://e3a9b532c60ade0f3c9976722af4b8dec00bb49b7ab98965e4e2782ea0c8834f" gracePeriod=45
Oct 30 21:46:30.117106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.117078    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main" containerID="cri-o://5c606bd708026adbb62f1a914fdf52679c959dfdcfdfee93ba5fd3d584215511" gracePeriod=45
Oct 30 21:46:30.387924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.387637    5786 generic.go:356] "Generic (PLEG): container finished" podID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerID="5c606bd708026adbb62f1a914fdf52679c959dfdcfdfee93ba5fd3d584215511" exitCode=0
Oct 30 21:46:30.387924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.387800    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" event={"ID":"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5","Type":"ContainerDied","Data":"5c606bd708026adbb62f1a914fdf52679c959dfdcfdfee93ba5fd3d584215511"}
Oct 30 21:46:30.389684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.389657    5786 generic.go:356] "Generic (PLEG): container finished" podID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerID="e3a9b532c60ade0f3c9976722af4b8dec00bb49b7ab98965e4e2782ea0c8834f" exitCode=0
Oct 30 21:46:30.389806 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.389733    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" event={"ID":"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16","Type":"ContainerDied","Data":"e3a9b532c60ade0f3c9976722af4b8dec00bb49b7ab98965e4e2782ea0c8834f"}
Oct 30 21:46:30.389915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.389834    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm" event={"ID":"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16","Type":"ContainerDied","Data":"db6db8939e7623b4622ed062541813de9d26ceb49244ff94337e3223beab4063"}
Oct 30 21:46:30.389915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.389860    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db6db8939e7623b4622ed062541813de9d26ceb49244ff94337e3223beab4063"
Oct 30 21:46:30.397670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.397649    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:46:30.415705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.415684    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-model-cache\") pod \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") "
Oct 30 21:46:30.415862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.415844    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-kube-api-access-mb657\" (UniqueName: \"kubernetes.io/projected/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-kube-api-access-mb657\") pod \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") "
Oct 30 21:46:30.415926 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.415879    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-model-cache" pod "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" (UID: "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:30.415965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.415950    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-dshm\") pod \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") "
Oct 30 21:46:30.416002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.415988    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tmp-dir\") pod \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") "
Oct 30 21:46:30.416058 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.416043    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-home\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-home\") pod \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") "
Oct 30 21:46:30.416139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.416125    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tls-certs\") pod \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\" (UID: \"5d55bbdf-7957-4ce8-96c7-9a45ed97dd16\") "
Oct 30 21:46:30.416322 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.416301    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tmp-dir" pod "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" (UID: "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:30.416434 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.416412    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-home" pod "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" (UID: "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:30.417074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.416930    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.417074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.416955    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.417074 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.416970    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.418433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.418407    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tls-certs" pod "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" (UID: "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:46:30.418516 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.418462    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-kube-api-access-mb657" pod "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" (UID: "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16"). InnerVolumeSpecName "kube-api-access-mb657". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:30.418516 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.418458    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-dshm" pod "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" (UID: "5d55bbdf-7957-4ce8-96c7-9a45ed97dd16"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:30.420664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.420649    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:46:30.522242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.520393    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-kube-api-access-zqrk4\" (UniqueName: \"kubernetes.io/projected/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-kube-api-access-zqrk4\") pod \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") "
Oct 30 21:46:30.522242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.520452    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tls-certs\" (UniqueName: \"kubernetes.io/secret/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tls-certs\") pod \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") "
Oct 30 21:46:30.522242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.520499    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-home\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-home\") pod \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") "
Oct 30 21:46:30.522242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.520554    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-dshm\") pod \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") "
Oct 30 21:46:30.522242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.520588    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-model-cache\") pod \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") "
Oct 30 21:46:30.522658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.522554    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tmp-dir\") pod \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\" (UID: \"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5\") "
Oct 30 21:46:30.524265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.523477    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.524265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.523507    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb657\" (UniqueName: \"kubernetes.io/projected/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-kube-api-access-mb657\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.524265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.523520    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.524265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.524041    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tmp-dir" pod "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" (UID: "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:30.524595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.524302    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-home" pod "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" (UID: "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:30.524898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.524870    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-model-cache" pod "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" (UID: "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:30.551313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.549789    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tls-certs" pod "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" (UID: "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:46:30.551313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.550004    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-kube-api-access-zqrk4" pod "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" (UID: "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5"). InnerVolumeSpecName "kube-api-access-zqrk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:30.551568 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.551320    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-dshm" pod "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" (UID: "1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:30.625421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.625347    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqrk4\" (UniqueName: \"kubernetes.io/projected/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-kube-api-access-zqrk4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.625421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.625371    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.625421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.625386    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.625421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.625398    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.625421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.625406    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:30.625421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:30.625414    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:31.397045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:31.396969    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"
Oct 30 21:46:31.397045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:31.396999    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"
Oct 30 21:46:31.397045 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:31.397014    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz" event={"ID":"1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5","Type":"ContainerDied","Data":"2ff4c754707caf0520a41832e8c01b153fc814b30faaeaec9fbea1361b78dab3"}
Oct 30 21:46:31.397641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:31.397055    5786 scope.go:122] "RemoveContainer" containerID="5c606bd708026adbb62f1a914fdf52679c959dfdcfdfee93ba5fd3d584215511"
Oct 30 21:46:31.558915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:31.558890    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"]
Oct 30 21:46:31.607695 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:31.607670    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/distinct-w3-9f52c3-kserve-6d8cb97c5b-b48jm"]
Oct 30 21:46:31.667247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:31.667219    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"]
Oct 30 21:46:31.670935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:31.670914    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9f52c3/simulator-w3-9f52c3-kserve-8f4bc958d-4bcqz"]
Oct 30 21:46:32.360373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:32.360345    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" path="/var/lib/kubelet/pods/1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5/volumes"
Oct 30 21:46:32.360809 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:32.360794    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" path="/var/lib/kubelet/pods/5d55bbdf-7957-4ce8-96c7-9a45ed97dd16/volumes"
Oct 30 21:46:32.827646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:32.827610    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s" podUID="70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:46:36.470973 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:36.470914    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"]
Oct 30 21:46:36.519443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:36.519415    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-815dab-vsk9g"]
Oct 30 21:46:38.359709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:38.359680    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39e9863c-1b01-437b-b794-130f4464a923" path="/var/lib/kubelet/pods/39e9863c-1b01-437b-b794-130f4464a923/volumes"
Oct 30 21:46:42.827331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:42.827292    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s" podUID="70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:46:47.909112 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:47.909086    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:46:48.038457 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.038428    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-plugins-config-volume\") pod \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\" (UID: \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\") "
Oct 30 21:46:48.038647 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.038628    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-kube-api-access-k9gcv\" (UniqueName: \"kubernetes.io/projected/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-kube-api-access-k9gcv\") pod \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\" (UID: \"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe\") "
Oct 30 21:46:48.038849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.038826    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-plugins-config-volume" pod "70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" (UID: "70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:48.039154 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.039138    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:48.040953 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.040921    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-kube-api-access-k9gcv" pod "70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" (UID: "70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe"). InnerVolumeSpecName "kube-api-access-k9gcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:48.140565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.140504    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9gcv\" (UniqueName: \"kubernetes.io/projected/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe-kube-api-access-k9gcv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:48.510452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.510423    5786 generic.go:356] "Generic (PLEG): container finished" podID="70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" containerID="d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9" exitCode=0
Oct 30 21:46:48.510601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.510464    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"
Oct 30 21:46:48.510601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.510504    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s" event={"ID":"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe","Type":"ContainerDied","Data":"d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9"}
Oct 30 21:46:48.510601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.510552    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s" event={"ID":"70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe","Type":"ContainerDied","Data":"05d3c85b711f161a0deff724a2757cc0dfeaedb199dbd7b798e71e1ce50e855c"}
Oct 30 21:46:48.510601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.510572    5786 scope.go:122] "RemoveContainer" containerID="d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9"
Oct 30 21:46:48.520193 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.520176    5786 scope.go:122] "RemoveContainer" containerID="d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9"
Oct 30 21:46:48.520483 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:46:48.520467    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9\": container with ID starting with d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9 not found: ID does not exist" containerID="d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9"
Oct 30 21:46:48.520533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.520491    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9"} err="failed to get container status \"d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9\": rpc error: code = NotFound desc = could not find container \"d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9\": container with ID starting with d02ed88518a56a9af510eaa517106925001aeea6a2c264ae02e5aa46ddb183b9 not found: ID does not exist"
Oct 30 21:46:48.693988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.690723    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"]
Oct 30 21:46:48.723338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:48.723301    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9f52c3-5c5d5f5c5-78n8s"]
Oct 30 21:46:50.359634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:50.359602    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" path="/var/lib/kubelet/pods/70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe/volumes"
Oct 30 21:46:52.355465 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:46:52.355419    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:46:59.754312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754227    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"]
Oct 30 21:46:59.754882 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754864    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5232f0a0-8b9a-4f57-8358-f09a856714ba" containerName="revoke-keys"
Oct 30 21:46:59.754882 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754882    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5232f0a0-8b9a-4f57-8358-f09a856714ba" containerName="revoke-keys"
Oct 30 21:46:59.755034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754898    5786 cpu_manager.go:414] "containerMap: removing container" podUID="70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" containerName="payload-pre-processing"
Oct 30 21:46:59.755034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754904    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" containerName="payload-pre-processing"
Oct 30 21:46:59.755034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754915    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main"
Oct 30 21:46:59.755034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754921    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main"
Oct 30 21:46:59.755034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754929    5786 cpu_manager.go:414] "containerMap: removing container" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main"
Oct 30 21:46:59.755034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.754934    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main"
Oct 30 21:46:59.755034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.755035    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5232f0a0-8b9a-4f57-8358-f09a856714ba" containerName="revoke-keys"
Oct 30 21:46:59.755474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.755047    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="70028dcc-a7a2-44e1-9dd2-c46c42bd8bbe" containerName="payload-pre-processing"
Oct 30 21:46:59.755474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.755052    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d55bbdf-7957-4ce8-96c7-9a45ed97dd16" containerName="main"
Oct 30 21:46:59.755474 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.755057    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bebc668-50a6-4eb6-824b-d2f6f0a3b0b5" containerName="main"
Oct 30 21:46:59.760057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.760038    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.814692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.814664    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"]
Oct 30 21:46:59.818000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.817970    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.818142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.818015    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rffx4\" (UniqueName: \"kubernetes.io/projected/3a186ff7-34e9-4338-829a-a951fd33d5f4-kube-api-access-rffx4\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.818142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.818109    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.859527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.859503    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:46:59.859647 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.859544    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:46:59.919927 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.919904    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.920073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.919935    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rffx4\" (UniqueName: \"kubernetes.io/projected/3a186ff7-34e9-4338-829a-a951fd33d5f4-kube-api-access-rffx4\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.920073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.919957    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.920624 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.920602    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.922476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.922447    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:46:59.976142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:46:59.976119    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rffx4\" (UniqueName: \"kubernetes.io/projected/3a186ff7-34e9-4338-829a-a951fd33d5f4-kube-api-access-rffx4\") pod \"maas-api-revoke-sub-309c89c5-d7rx7\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:47:00.073972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:00.073925    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:47:00.450811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:00.450782    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"]
Oct 30 21:47:00.451882 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:47:00.451858    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a186ff7_34e9_4338_829a_a951fd33d5f4.slice/crio-5dc87ad548b35d91e616529e0d30291f12ab3d5b00cfd1e56c502e71cab96801 WatchSource:0}: Error finding container 5dc87ad548b35d91e616529e0d30291f12ab3d5b00cfd1e56c502e71cab96801: Status 404 returned error can't find the container with id 5dc87ad548b35d91e616529e0d30291f12ab3d5b00cfd1e56c502e71cab96801
Oct 30 21:47:00.596241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:00.596187    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7" event={"ID":"3a186ff7-34e9-4338-829a-a951fd33d5f4","Type":"ContainerStarted","Data":"5dc87ad548b35d91e616529e0d30291f12ab3d5b00cfd1e56c502e71cab96801"}
Oct 30 21:47:01.604470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:01.604391    5786 generic.go:356] "Generic (PLEG): container finished" podID="3a186ff7-34e9-4338-829a-a951fd33d5f4" containerID="38aa8429f90beb03f2ff79505c94d5e24cffaaa20123540b9a6c0f0d8f692684" exitCode=0
Oct 30 21:47:01.604804 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:01.604478    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7" event={"ID":"3a186ff7-34e9-4338-829a-a951fd33d5f4","Type":"ContainerDied","Data":"38aa8429f90beb03f2ff79505c94d5e24cffaaa20123540b9a6c0f0d8f692684"}
Oct 30 21:47:02.768899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.768870    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:47:02.852402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.852377    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-xks-ca\") pod \"3a186ff7-34e9-4338-829a-a951fd33d5f4\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") "
Oct 30 21:47:02.852518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.852461    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-openshift-ca\") pod \"3a186ff7-34e9-4338-829a-a951fd33d5f4\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") "
Oct 30 21:47:02.852617 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.852605    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a186ff7-34e9-4338-829a-a951fd33d5f4-kube-api-access-rffx4\" (UniqueName: \"kubernetes.io/projected/3a186ff7-34e9-4338-829a-a951fd33d5f4-kube-api-access-rffx4\") pod \"3a186ff7-34e9-4338-829a-a951fd33d5f4\" (UID: \"3a186ff7-34e9-4338-829a-a951fd33d5f4\") "
Oct 30 21:47:02.852834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.852810    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-openshift-ca" pod "3a186ff7-34e9-4338-829a-a951fd33d5f4" (UID: "3a186ff7-34e9-4338-829a-a951fd33d5f4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:02.853026 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.853003    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:02.854548 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.854528    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-xks-ca" pod "3a186ff7-34e9-4338-829a-a951fd33d5f4" (UID: "3a186ff7-34e9-4338-829a-a951fd33d5f4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:02.854653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.854639    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a186ff7-34e9-4338-829a-a951fd33d5f4-kube-api-access-rffx4" pod "3a186ff7-34e9-4338-829a-a951fd33d5f4" (UID: "3a186ff7-34e9-4338-829a-a951fd33d5f4"). InnerVolumeSpecName "kube-api-access-rffx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:02.954725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.954693    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rffx4\" (UniqueName: \"kubernetes.io/projected/3a186ff7-34e9-4338-829a-a951fd33d5f4-kube-api-access-rffx4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:02.954725 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:02.954726    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a186ff7-34e9-4338-829a-a951fd33d5f4-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:03.619386 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:03.619354    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"
Oct 30 21:47:03.619572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:03.619383    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7" event={"ID":"3a186ff7-34e9-4338-829a-a951fd33d5f4","Type":"ContainerDied","Data":"5dc87ad548b35d91e616529e0d30291f12ab3d5b00cfd1e56c502e71cab96801"}
Oct 30 21:47:03.619572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:03.619437    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5dc87ad548b35d91e616529e0d30291f12ab3d5b00cfd1e56c502e71cab96801"
Oct 30 21:47:04.681873 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.681843    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"]
Oct 30 21:47:04.682496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.682483    5786 cpu_manager.go:414] "containerMap: removing container" podUID="3a186ff7-34e9-4338-829a-a951fd33d5f4" containerName="revoke-keys"
Oct 30 21:47:04.682547 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.682498    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a186ff7-34e9-4338-829a-a951fd33d5f4" containerName="revoke-keys"
Oct 30 21:47:04.682607 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.682599    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a186ff7-34e9-4338-829a-a951fd33d5f4" containerName="revoke-keys"
Oct 30 21:47:04.686312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.686296    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.773483 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.773452    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.773643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.773489    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.773710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.773646    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2qbv\" (UniqueName: \"kubernetes.io/projected/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-kube-api-access-w2qbv\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.787825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.787803    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:47:04.787965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.787949    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:47:04.791424 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.791401    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"]
Oct 30 21:47:04.875012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.874986    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.875109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.875015    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.875109 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.875053    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2qbv\" (UniqueName: \"kubernetes.io/projected/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-kube-api-access-w2qbv\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.875634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.875610    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.877412 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.877392    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.930949 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.930928    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2qbv\" (UniqueName: \"kubernetes.io/projected/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-kube-api-access-w2qbv\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:04.999908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:04.999858    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:05.377267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:05.377238    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"]
Oct 30 21:47:05.378841 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:47:05.378811    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84cbdaea_2db6_494b_9fd4_bb5a4e056e77.slice/crio-8da3635b214b56af92dcfa6f8fb5e454e2c3c26a2461df5ab10e7628d232f307 WatchSource:0}: Error finding container 8da3635b214b56af92dcfa6f8fb5e454e2c3c26a2461df5ab10e7628d232f307: Status 404 returned error can't find the container with id 8da3635b214b56af92dcfa6f8fb5e454e2c3c26a2461df5ab10e7628d232f307
Oct 30 21:47:05.633403 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:05.633334    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s" event={"ID":"84cbdaea-2db6-494b-9fd4-bb5a4e056e77","Type":"ContainerStarted","Data":"8da3635b214b56af92dcfa6f8fb5e454e2c3c26a2461df5ab10e7628d232f307"}
Oct 30 21:47:06.641472 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:06.641396    5786 generic.go:356] "Generic (PLEG): container finished" podID="84cbdaea-2db6-494b-9fd4-bb5a4e056e77" containerID="1f17a36ddf2021b3e891d22139a7017a4249c9e7632f163c6c5579608fae8efe" exitCode=0
Oct 30 21:47:06.641840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:06.641474    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s" event={"ID":"84cbdaea-2db6-494b-9fd4-bb5a4e056e77","Type":"ContainerDied","Data":"1f17a36ddf2021b3e891d22139a7017a4249c9e7632f163c6c5579608fae8efe"}
Oct 30 21:47:07.806086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:07.806066    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:07.907887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:07.907832    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-kube-api-access-w2qbv\" (UniqueName: \"kubernetes.io/projected/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-kube-api-access-w2qbv\") pod \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") "
Oct 30 21:47:07.907887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:07.907870    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-openshift-ca\") pod \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") "
Oct 30 21:47:07.908066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:07.907944    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-xks-ca\") pod \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\" (UID: \"84cbdaea-2db6-494b-9fd4-bb5a4e056e77\") "
Oct 30 21:47:07.908248 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:07.908224    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-openshift-ca" pod "84cbdaea-2db6-494b-9fd4-bb5a4e056e77" (UID: "84cbdaea-2db6-494b-9fd4-bb5a4e056e77"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:07.908445 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:07.908430    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:07.909905 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:07.909884    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-kube-api-access-w2qbv" pod "84cbdaea-2db6-494b-9fd4-bb5a4e056e77" (UID: "84cbdaea-2db6-494b-9fd4-bb5a4e056e77"). InnerVolumeSpecName "kube-api-access-w2qbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:07.910002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:07.909978    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-xks-ca" pod "84cbdaea-2db6-494b-9fd4-bb5a4e056e77" (UID: "84cbdaea-2db6-494b-9fd4-bb5a4e056e77"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:08.009912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:08.009885    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:08.009912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:08.009912    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2qbv\" (UniqueName: \"kubernetes.io/projected/84cbdaea-2db6-494b-9fd4-bb5a4e056e77-kube-api-access-w2qbv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:08.655862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:08.655792    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s" event={"ID":"84cbdaea-2db6-494b-9fd4-bb5a4e056e77","Type":"ContainerDied","Data":"8da3635b214b56af92dcfa6f8fb5e454e2c3c26a2461df5ab10e7628d232f307"}
Oct 30 21:47:08.655862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:08.655835    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8da3635b214b56af92dcfa6f8fb5e454e2c3c26a2461df5ab10e7628d232f307"
Oct 30 21:47:08.655862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:08.655800    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"
Oct 30 21:47:16.228319 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:16.228283    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"]
Oct 30 21:47:16.340777 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:16.337863    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"]
Oct 30 21:47:21.229139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.229075    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j" podUID="d6bfa334-afee-4d0d-b43b-bf860dae5f1d" containerName="payload-processing" containerID="cri-o://cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf" gracePeriod=25
Oct 30 21:47:21.338313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.338247    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c" podUID="bc71b1a2-432d-48ad-a8a9-760c530e032d" containerName="payload-pre-processing" containerID="cri-o://62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456" gracePeriod=25
Oct 30 21:47:21.483066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.483006    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:47:21.576384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.576355    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-kube-api-access-wsjzk\" (UniqueName: \"kubernetes.io/projected/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-kube-api-access-wsjzk\") pod \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\" (UID: \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\") "
Oct 30 21:47:21.576610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.576593    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-plugins-config-volume\") pod \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\" (UID: \"d6bfa334-afee-4d0d-b43b-bf860dae5f1d\") "
Oct 30 21:47:21.576947 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.576924    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-plugins-config-volume" pod "d6bfa334-afee-4d0d-b43b-bf860dae5f1d" (UID: "d6bfa334-afee-4d0d-b43b-bf860dae5f1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:21.578693 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.578656    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-kube-api-access-wsjzk" pod "d6bfa334-afee-4d0d-b43b-bf860dae5f1d" (UID: "d6bfa334-afee-4d0d-b43b-bf860dae5f1d"). InnerVolumeSpecName "kube-api-access-wsjzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:21.601138 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.601115    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:47:21.677992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.677969    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc71b1a2-432d-48ad-a8a9-760c530e032d-kube-api-access-9qndl\" (UniqueName: \"kubernetes.io/projected/bc71b1a2-432d-48ad-a8a9-760c530e032d-kube-api-access-9qndl\") pod \"bc71b1a2-432d-48ad-a8a9-760c530e032d\" (UID: \"bc71b1a2-432d-48ad-a8a9-760c530e032d\") "
Oct 30 21:47:21.678131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.678119    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc71b1a2-432d-48ad-a8a9-760c530e032d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc71b1a2-432d-48ad-a8a9-760c530e032d-plugins-config-volume\") pod \"bc71b1a2-432d-48ad-a8a9-760c530e032d\" (UID: \"bc71b1a2-432d-48ad-a8a9-760c530e032d\") "
Oct 30 21:47:21.678461 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.678449    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:21.678510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.678463    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsjzk\" (UniqueName: \"kubernetes.io/projected/d6bfa334-afee-4d0d-b43b-bf860dae5f1d-kube-api-access-wsjzk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:21.678510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.678487    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc71b1a2-432d-48ad-a8a9-760c530e032d-plugins-config-volume" pod "bc71b1a2-432d-48ad-a8a9-760c530e032d" (UID: "bc71b1a2-432d-48ad-a8a9-760c530e032d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:21.680162 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.680146    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc71b1a2-432d-48ad-a8a9-760c530e032d-kube-api-access-9qndl" pod "bc71b1a2-432d-48ad-a8a9-760c530e032d" (UID: "bc71b1a2-432d-48ad-a8a9-760c530e032d"). InnerVolumeSpecName "kube-api-access-9qndl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:21.756586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.756520    5786 generic.go:356] "Generic (PLEG): container finished" podID="d6bfa334-afee-4d0d-b43b-bf860dae5f1d" containerID="cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf" exitCode=0
Oct 30 21:47:21.756586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.756561    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"
Oct 30 21:47:21.756749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.756597    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j" event={"ID":"d6bfa334-afee-4d0d-b43b-bf860dae5f1d","Type":"ContainerDied","Data":"cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf"}
Oct 30 21:47:21.756749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.756651    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j" event={"ID":"d6bfa334-afee-4d0d-b43b-bf860dae5f1d","Type":"ContainerDied","Data":"cdf180c7abf7572ae4a30da193746fc932777e5d44487073496ab041feebb7ee"}
Oct 30 21:47:21.756749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.756678    5786 scope.go:122] "RemoveContainer" containerID="cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf"
Oct 30 21:47:21.758338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.758319    5786 generic.go:356] "Generic (PLEG): container finished" podID="bc71b1a2-432d-48ad-a8a9-760c530e032d" containerID="62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456" exitCode=0
Oct 30 21:47:21.758431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.758361    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"
Oct 30 21:47:21.758431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.758376    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c" event={"ID":"bc71b1a2-432d-48ad-a8a9-760c530e032d","Type":"ContainerDied","Data":"62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456"}
Oct 30 21:47:21.758431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.758410    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c" event={"ID":"bc71b1a2-432d-48ad-a8a9-760c530e032d","Type":"ContainerDied","Data":"edb07943860fab5c9d20e6cc1917b250fccb7bd7edfdaf7982da63b3fd2e1828"}
Oct 30 21:47:21.767122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.767102    5786 scope.go:122] "RemoveContainer" containerID="cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf"
Oct 30 21:47:21.767391 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:47:21.767369    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf\": container with ID starting with cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf not found: ID does not exist" containerID="cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf"
Oct 30 21:47:21.767449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.767396    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf"} err="failed to get container status \"cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf\": rpc error: code = NotFound desc = could not find container \"cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf\": container with ID starting with cdfd2c84fdbe14590d62dd7c9953421bc66eca44bf38e00cb86d82527748a1cf not found: ID does not exist"
Oct 30 21:47:21.767449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.767416    5786 scope.go:122] "RemoveContainer" containerID="62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456"
Oct 30 21:47:21.776615 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.776597    5786 scope.go:122] "RemoveContainer" containerID="62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456"
Oct 30 21:47:21.776903 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:47:21.776884    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456\": container with ID starting with 62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456 not found: ID does not exist" containerID="62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456"
Oct 30 21:47:21.776975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.776908    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456"} err="failed to get container status \"62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456\": rpc error: code = NotFound desc = could not find container \"62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456\": container with ID starting with 62567e628606e0e6e14513d5a2d7e4c9d1375eb2876314478ef88823e2140456 not found: ID does not exist"
Oct 30 21:47:21.779660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.779639    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc71b1a2-432d-48ad-a8a9-760c530e032d-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:21.779660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.779660    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qndl\" (UniqueName: \"kubernetes.io/projected/bc71b1a2-432d-48ad-a8a9-760c530e032d-kube-api-access-9qndl\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:21.919270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.919248    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"]
Oct 30 21:47:21.922342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:21.922323    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99003c-584cdb6999-pbf4j"]
Oct 30 21:47:22.029906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:22.029852    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"]
Oct 30 21:47:22.033510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:22.033491    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-99003c-584fd5dc67-rfn7c"]
Oct 30 21:47:22.354684 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:47:22.354614    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:47:22.359640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:22.359617    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc71b1a2-432d-48ad-a8a9-760c530e032d" path="/var/lib/kubelet/pods/bc71b1a2-432d-48ad-a8a9-760c530e032d/volumes"
Oct 30 21:47:22.359986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:22.359973    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6bfa334-afee-4d0d-b43b-bf860dae5f1d" path="/var/lib/kubelet/pods/d6bfa334-afee-4d0d-b43b-bf860dae5f1d/volumes"
Oct 30 21:47:26.905749 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:26.905714    5786 scope.go:122] "RemoveContainer" containerID="e3a9b532c60ade0f3c9976722af4b8dec00bb49b7ab98965e4e2782ea0c8834f"
Oct 30 21:47:39.271855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.271827    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"]
Oct 30 21:47:39.272276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.272057    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp" podUID="c8d8457a-0a0c-4cdf-864a-cf5209912d34" containerName="maas-api" containerID="cri-o://b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99" gracePeriod=30
Oct 30 21:47:39.564033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.564010    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:47:39.733692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.733665    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8d8457a-0a0c-4cdf-864a-cf5209912d34-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d8457a-0a0c-4cdf-864a-cf5209912d34-openshift-service-ca\") pod \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") "
Oct 30 21:47:39.733882 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.733822    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls\") pod \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") "
Oct 30 21:47:39.733882 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.733858    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8d8457a-0a0c-4cdf-864a-cf5209912d34-kube-api-access-pm8h9\" (UniqueName: \"kubernetes.io/projected/c8d8457a-0a0c-4cdf-864a-cf5209912d34-kube-api-access-pm8h9\") pod \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") "
Oct 30 21:47:39.734005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.733895    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-metrics-certs\") pod \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\" (UID: \"c8d8457a-0a0c-4cdf-864a-cf5209912d34\") "
Oct 30 21:47:39.734108 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.734060    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8d8457a-0a0c-4cdf-864a-cf5209912d34-openshift-service-ca" pod "c8d8457a-0a0c-4cdf-864a-cf5209912d34" (UID: "c8d8457a-0a0c-4cdf-864a-cf5209912d34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:39.734436 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.734414    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d8457a-0a0c-4cdf-864a-cf5209912d34-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:39.736326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.736301    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8d8457a-0a0c-4cdf-864a-cf5209912d34-kube-api-access-pm8h9" pod "c8d8457a-0a0c-4cdf-864a-cf5209912d34" (UID: "c8d8457a-0a0c-4cdf-864a-cf5209912d34"). InnerVolumeSpecName "kube-api-access-pm8h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:39.736811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.736787    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls" pod "c8d8457a-0a0c-4cdf-864a-cf5209912d34" (UID: "c8d8457a-0a0c-4cdf-864a-cf5209912d34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:39.736944 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.736801    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-metrics-certs" pod "c8d8457a-0a0c-4cdf-864a-cf5209912d34" (UID: "c8d8457a-0a0c-4cdf-864a-cf5209912d34"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:39.836283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.835835    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:39.836283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.835879    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm8h9\" (UniqueName: \"kubernetes.io/projected/c8d8457a-0a0c-4cdf-864a-cf5209912d34-kube-api-access-pm8h9\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:39.836283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.835894    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8d8457a-0a0c-4cdf-864a-cf5209912d34-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:39.897435 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.897402    5786 generic.go:356] "Generic (PLEG): container finished" podID="c8d8457a-0a0c-4cdf-864a-cf5209912d34" containerID="b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99" exitCode=0
Oct 30 21:47:39.897611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.897441    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"
Oct 30 21:47:39.897611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.897457    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp" event={"ID":"c8d8457a-0a0c-4cdf-864a-cf5209912d34","Type":"ContainerDied","Data":"b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99"}
Oct 30 21:47:39.897611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.897504    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp" event={"ID":"c8d8457a-0a0c-4cdf-864a-cf5209912d34","Type":"ContainerDied","Data":"7362e9b304c76b39060d1aaf6e8167d6f414c2e398abd4f6b5627af1c9ba894a"}
Oct 30 21:47:39.897611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.897527    5786 scope.go:122] "RemoveContainer" containerID="b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99"
Oct 30 21:47:39.908406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.908391    5786 scope.go:122] "RemoveContainer" containerID="b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99"
Oct 30 21:47:39.908635 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:47:39.908620    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99\": container with ID starting with b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99 not found: ID does not exist" containerID="b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99"
Oct 30 21:47:39.908690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:39.908644    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99"} err="failed to get container status \"b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99\": rpc error: code = NotFound desc = could not find container \"b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99\": container with ID starting with b089c55c8be2a32a621c2805d6f681a9a8c20d72d6763a97aebd3f2b5ea7da99 not found: ID does not exist"
Oct 30 21:47:40.059347 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:40.059320    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"]
Oct 30 21:47:40.063258 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:40.063235    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99003c-6989774455-dfrbp"]
Oct 30 21:47:40.360913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:40.360880    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8d8457a-0a0c-4cdf-864a-cf5209912d34" path="/var/lib/kubelet/pods/c8d8457a-0a0c-4cdf-864a-cf5209912d34/volumes"
Oct 30 21:47:41.381578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:41.381548    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"]
Oct 30 21:47:41.430425 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:41.430399    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99003c-hc89s"]
Oct 30 21:47:42.359750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:42.359721    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84cbdaea-2db6-494b-9fd4-bb5a4e056e77" path="/var/lib/kubelet/pods/84cbdaea-2db6-494b-9fd4-bb5a4e056e77/volumes"
Oct 30 21:47:49.395839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.395799    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"]
Oct 30 21:47:49.396444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396429    5786 cpu_manager.go:414] "containerMap: removing container" podUID="bc71b1a2-432d-48ad-a8a9-760c530e032d" containerName="payload-pre-processing"
Oct 30 21:47:49.396495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396447    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc71b1a2-432d-48ad-a8a9-760c530e032d" containerName="payload-pre-processing"
Oct 30 21:47:49.396495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396462    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d6bfa334-afee-4d0d-b43b-bf860dae5f1d" containerName="payload-processing"
Oct 30 21:47:49.396495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396470    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6bfa334-afee-4d0d-b43b-bf860dae5f1d" containerName="payload-processing"
Oct 30 21:47:49.396495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396492    5786 cpu_manager.go:414] "containerMap: removing container" podUID="84cbdaea-2db6-494b-9fd4-bb5a4e056e77" containerName="revoke-keys"
Oct 30 21:47:49.396644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396497    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84cbdaea-2db6-494b-9fd4-bb5a4e056e77" containerName="revoke-keys"
Oct 30 21:47:49.396644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396518    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c8d8457a-0a0c-4cdf-864a-cf5209912d34" containerName="maas-api"
Oct 30 21:47:49.396644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396522    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8d8457a-0a0c-4cdf-864a-cf5209912d34" containerName="maas-api"
Oct 30 21:47:49.396644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396622    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="84cbdaea-2db6-494b-9fd4-bb5a4e056e77" containerName="revoke-keys"
Oct 30 21:47:49.396644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396634    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8d8457a-0a0c-4cdf-864a-cf5209912d34" containerName="maas-api"
Oct 30 21:47:49.396644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396644    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc71b1a2-432d-48ad-a8a9-760c530e032d" containerName="payload-pre-processing"
Oct 30 21:47:49.396833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.396650    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6bfa334-afee-4d0d-b43b-bf860dae5f1d" containerName="payload-processing"
Oct 30 21:47:49.400359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.400341    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.453760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.453730    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:47:49.457646 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.457617    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.457783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.457767    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf9wx\" (UniqueName: \"kubernetes.io/projected/c6d5a236-8080-4cfb-a01e-0b4578d85c52-kube-api-access-vf9wx\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.457865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.457819    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.502797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.502779    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:47:49.505912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.505892    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"]
Oct 30 21:47:49.559212 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.559185    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf9wx\" (UniqueName: \"kubernetes.io/projected/c6d5a236-8080-4cfb-a01e-0b4578d85c52-kube-api-access-vf9wx\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.559318 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.559239    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.559382 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.559368    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.559958 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.559940    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.561623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.561598    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.631232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.630341    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf9wx\" (UniqueName: \"kubernetes.io/projected/c6d5a236-8080-4cfb-a01e-0b4578d85c52-kube-api-access-vf9wx\") pod \"maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:49.714634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:49.714598    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:50.090924 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:50.090899    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"]
Oct 30 21:47:50.092503 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:47:50.092473    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d5a236_8080_4cfb_a01e_0b4578d85c52.slice/crio-51d7dcd562fd0f490ab1b1e9a75c4741d03dd82e386e463595cb1dabaef0fe4e WatchSource:0}: Error finding container 51d7dcd562fd0f490ab1b1e9a75c4741d03dd82e386e463595cb1dabaef0fe4e: Status 404 returned error can't find the container with id 51d7dcd562fd0f490ab1b1e9a75c4741d03dd82e386e463595cb1dabaef0fe4e
Oct 30 21:47:50.974596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:50.974569    5786 generic.go:356] "Generic (PLEG): container finished" podID="c6d5a236-8080-4cfb-a01e-0b4578d85c52" containerID="6b9edf4bdd35a3528a766c68eafc4f7026a27609b54679543848a2f127dfae53" exitCode=0
Oct 30 21:47:50.974948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:50.974611    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj" event={"ID":"c6d5a236-8080-4cfb-a01e-0b4578d85c52","Type":"ContainerDied","Data":"6b9edf4bdd35a3528a766c68eafc4f7026a27609b54679543848a2f127dfae53"}
Oct 30 21:47:50.974948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:50.974665    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj" event={"ID":"c6d5a236-8080-4cfb-a01e-0b4578d85c52","Type":"ContainerStarted","Data":"51d7dcd562fd0f490ab1b1e9a75c4741d03dd82e386e463595cb1dabaef0fe4e"}
Oct 30 21:47:52.052718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.052686    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"]
Oct 30 21:47:52.101136 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.101106    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04c9cda6-b8c8w"]
Oct 30 21:47:52.122894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.122875    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:52.191324 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.191298    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-openshift-ca\") pod \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") "
Oct 30 21:47:52.191453 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.191438    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-xks-ca\") pod \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") "
Oct 30 21:47:52.191528 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.191474    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6d5a236-8080-4cfb-a01e-0b4578d85c52-kube-api-access-vf9wx\" (UniqueName: \"kubernetes.io/projected/c6d5a236-8080-4cfb-a01e-0b4578d85c52-kube-api-access-vf9wx\") pod \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\" (UID: \"c6d5a236-8080-4cfb-a01e-0b4578d85c52\") "
Oct 30 21:47:52.191645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.191624    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-openshift-ca" pod "c6d5a236-8080-4cfb-a01e-0b4578d85c52" (UID: "c6d5a236-8080-4cfb-a01e-0b4578d85c52"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:52.191921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.191903    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:52.193544 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.193526    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-xks-ca" pod "c6d5a236-8080-4cfb-a01e-0b4578d85c52" (UID: "c6d5a236-8080-4cfb-a01e-0b4578d85c52"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:52.193642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.193620    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d5a236-8080-4cfb-a01e-0b4578d85c52-kube-api-access-vf9wx" pod "c6d5a236-8080-4cfb-a01e-0b4578d85c52" (UID: "c6d5a236-8080-4cfb-a01e-0b4578d85c52"). InnerVolumeSpecName "kube-api-access-vf9wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:52.293098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.293041    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6d5a236-8080-4cfb-a01e-0b4578d85c52-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:52.293098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.293066    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf9wx\" (UniqueName: \"kubernetes.io/projected/c6d5a236-8080-4cfb-a01e-0b4578d85c52-kube-api-access-vf9wx\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:47:52.382692 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.382661    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="117d7cfc-33d9-4d6c-b84f-dcedead30bc9" path="/var/lib/kubelet/pods/117d7cfc-33d9-4d6c-b84f-dcedead30bc9/volumes"
Oct 30 21:47:52.384346 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:47:52.384312    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:47:52.505047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.505016    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"]
Oct 30 21:47:52.507730 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.507710    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-9f52c3-xd46n"]
Oct 30 21:47:52.988896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.988873    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"
Oct 30 21:47:52.988896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.988880    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj" event={"ID":"c6d5a236-8080-4cfb-a01e-0b4578d85c52","Type":"ContainerDied","Data":"51d7dcd562fd0f490ab1b1e9a75c4741d03dd82e386e463595cb1dabaef0fe4e"}
Oct 30 21:47:52.989104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:52.988921    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51d7dcd562fd0f490ab1b1e9a75c4741d03dd82e386e463595cb1dabaef0fe4e"
Oct 30 21:47:53.097484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:53.097454    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"]
Oct 30 21:47:53.100018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:53.099996    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4693dc-b649w"]
Oct 30 21:47:53.875612 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:53.875581    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"]
Oct 30 21:47:53.925510 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:53.925481    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"]
Oct 30 21:47:54.360362 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:54.360329    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5232f0a0-8b9a-4f57-8358-f09a856714ba" path="/var/lib/kubelet/pods/5232f0a0-8b9a-4f57-8358-f09a856714ba/volumes"
Oct 30 21:47:54.360760 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:54.360746    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a96bdf00-65f8-4081-9b0e-7d4b6345b7be" path="/var/lib/kubelet/pods/a96bdf00-65f8-4081-9b0e-7d4b6345b7be/volumes"
Oct 30 21:47:59.559350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:59.559318    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"]
Oct 30 21:47:59.559861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:59.559548    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy" containerID="cri-o://8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6" gracePeriod=30
Oct 30 21:47:59.574331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:59.574301    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"]
Oct 30 21:47:59.574519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:47:59.574499    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy" containerID="cri-o://92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31" gracePeriod=30
Oct 30 21:48:00.947882 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:00.947844    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"]
Oct 30 21:48:02.439736 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:02.439696    5786 patch_prober.go:29] interesting pod/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:48:02.440278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:02.439774    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:48:04.837016 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.836953    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:48:04.864831 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.864809    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:48:04.969867 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.969843    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-istio-token\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-istio-token\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.969888    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-envoy\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.970029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.969907    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-envoy\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.969938    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-token\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-token\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.970029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.969988    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-certs\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.970029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970019    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6e7280e-7606-4185-913b-9142f7a88f56-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e7280e-7606-4185-913b-9142f7a88f56-istiod-ca-cert\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970072    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-podinfo\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.970360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970113    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-credential-socket\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.970360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970144    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-socket\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.970360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970197    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-data\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.970360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970257    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-kube-api-access-7j426\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-kube-api-access-7j426\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970308    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-certs" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.970360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970348    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-credential-socket\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970391    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c6e7280e-7606-4185-913b-9142f7a88f56-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e7280e-7606-4185-913b-9142f7a88f56-istio-podinfo\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970425    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-data\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970453    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-socket\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970511    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-certs\") pod \"c6e7280e-7606-4185-913b-9142f7a88f56\" (UID: \"c6e7280e-7606-4185-913b-9142f7a88f56\") "
Oct 30 21:48:04.970881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970543    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-kube-api-access-9ndqc\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-kube-api-access-9ndqc\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.970881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970601    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/969f882f-d0de-48b8-839d-e33a4fd911fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f882f-d0de-48b8-839d-e33a4fd911fe-istiod-ca-cert\") pod \"969f882f-d0de-48b8-839d-e33a4fd911fe\" (UID: \"969f882f-d0de-48b8-839d-e33a4fd911fe\") "
Oct 30 21:48:04.971199 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.971111    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:04.971290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970418    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6e7280e-7606-4185-913b-9142f7a88f56-istiod-ca-cert" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:04.971348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970485    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-credential-socket" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.971405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.971342    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-certs" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.971405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970496    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-socket" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.971405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970671    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-data" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.971405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.970836    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-credential-socket" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.971405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.971175    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-data" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.971677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.971457    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-socket" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.971677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.971484    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969f882f-d0de-48b8-839d-e33a4fd911fe-istiod-ca-cert" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:04.972845 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.972818    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-envoy" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.972979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.972959    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-token" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:04.973400 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.973379    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-kube-api-access-7j426" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "kube-api-access-7j426". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:04.973491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.973400    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-istio-token" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:04.973621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.973593    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-envoy" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:04.973671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.973651    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6e7280e-7606-4185-913b-9142f7a88f56-istio-podinfo" pod "c6e7280e-7606-4185-913b-9142f7a88f56" (UID: "c6e7280e-7606-4185-913b-9142f7a88f56"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:48:04.973741 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.973720    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-podinfo" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:48:04.974812 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:04.974798    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-kube-api-access-9ndqc" pod "969f882f-d0de-48b8-839d-e33a4fd911fe" (UID: "969f882f-d0de-48b8-839d-e33a4fd911fe"). InnerVolumeSpecName "kube-api-access-9ndqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:05.073134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073101    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f882f-d0de-48b8-839d-e33a4fd911fe-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073139    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073153    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073166    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073179    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073194    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e7280e-7606-4185-913b-9142f7a88f56-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073225    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073239    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073252    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073264    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f882f-d0de-48b8-839d-e33a4fd911fe-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073276    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j426\" (UniqueName: \"kubernetes.io/projected/c6e7280e-7606-4185-913b-9142f7a88f56-kube-api-access-7j426\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073291    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073303    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e7280e-7606-4185-913b-9142f7a88f56-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073316    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073329    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073344    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e7280e-7606-4185-913b-9142f7a88f56-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.073827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.073357    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ndqc\" (UniqueName: \"kubernetes.io/projected/969f882f-d0de-48b8-839d-e33a4fd911fe-kube-api-access-9ndqc\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:05.078333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.078309    5786 generic.go:356] "Generic (PLEG): container finished" podID="c6e7280e-7606-4185-913b-9142f7a88f56" containerID="92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31" exitCode=0
Oct 30 21:48:05.078452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.078355    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"
Oct 30 21:48:05.078452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.078391    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" event={"ID":"c6e7280e-7606-4185-913b-9142f7a88f56","Type":"ContainerDied","Data":"92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31"}
Oct 30 21:48:05.078452 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.078446    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl" event={"ID":"c6e7280e-7606-4185-913b-9142f7a88f56","Type":"ContainerDied","Data":"c03655ab941eb3baaa9fac05b929418fb861bbdaf3bd145d4e4cd3e704b9bdfc"}
Oct 30 21:48:05.078602 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.078474    5786 scope.go:122] "RemoveContainer" containerID="92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31"
Oct 30 21:48:05.079868 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.079843    5786 generic.go:356] "Generic (PLEG): container finished" podID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerID="8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6" exitCode=0
Oct 30 21:48:05.079988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.079893    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"
Oct 30 21:48:05.079988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.079925    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" event={"ID":"969f882f-d0de-48b8-839d-e33a4fd911fe","Type":"ContainerDied","Data":"8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6"}
Oct 30 21:48:05.080091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.079986    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n" event={"ID":"969f882f-d0de-48b8-839d-e33a4fd911fe","Type":"ContainerDied","Data":"1cc2f21918e7053de06961f63a354e6c47970dbb7456fe5e1c1c2f3a8e30193b"}
Oct 30 21:48:05.090005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.089990    5786 scope.go:122] "RemoveContainer" containerID="92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31"
Oct 30 21:48:05.090310 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:48:05.090291    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31\": container with ID starting with 92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31 not found: ID does not exist" containerID="92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31"
Oct 30 21:48:05.090409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.090320    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31"} err="failed to get container status \"92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31\": rpc error: code = NotFound desc = could not find container \"92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31\": container with ID starting with 92342efadd4f8d0de79ae9bc570eb619855c59759f2821784b2e1b742bb32c31 not found: ID does not exist"
Oct 30 21:48:05.090409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.090343    5786 scope.go:122] "RemoveContainer" containerID="8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6"
Oct 30 21:48:05.100130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.100064    5786 scope.go:122] "RemoveContainer" containerID="8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6"
Oct 30 21:48:05.100563 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:48:05.100535    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6\": container with ID starting with 8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6 not found: ID does not exist" containerID="8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6"
Oct 30 21:48:05.100676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.100570    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6"} err="failed to get container status \"8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6\": rpc error: code = NotFound desc = could not find container \"8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6\": container with ID starting with 8ea24a9c550bd109d0abe2ef2b55d396a7d3c71f2efcc43d44c3e11aa9b5c0d6 not found: ID does not exist"
Oct 30 21:48:05.229700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.229665    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"]
Oct 30 21:48:05.233008 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.232983    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-2gxhl"]
Oct 30 21:48:05.339672 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.339643    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"]
Oct 30 21:48:05.343849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.343797    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f52c3-openshift-default-df5ffd65f-lr47n"]
Oct 30 21:48:05.948123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:05.948095    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24" podUID="bb5f2a0f-9f81-4f88-8e65-bb4499139f87" containerName="payload-processing" containerID="cri-o://f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e" gracePeriod=25
Oct 30 21:48:06.371052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:06.370967    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" path="/var/lib/kubelet/pods/969f882f-d0de-48b8-839d-e33a4fd911fe/volumes"
Oct 30 21:48:06.371640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:06.371622    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" path="/var/lib/kubelet/pods/c6e7280e-7606-4185-913b-9142f7a88f56/volumes"
Oct 30 21:48:08.897059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:08.897019    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" containerID="cri-o://eea172e9f44880ac1a70b06c92c382680f201647f98c8a3cc8d54469ad74836f" gracePeriod=45
Oct 30 21:48:08.945253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:08.945196    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main" containerID="cri-o://a1761962f7a8110950049d3b00744c699ad3cf1a022f1a92a8eda52f49634d46" gracePeriod=45
Oct 30 21:48:08.945656 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:08.945628    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.82:8000/ready\": dial tcp 10.134.0.82:8000: connect: connection refused"
Oct 30 21:48:09.107923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.107744    5786 generic.go:356] "Generic (PLEG): container finished" podID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerID="a1761962f7a8110950049d3b00744c699ad3cf1a022f1a92a8eda52f49634d46" exitCode=0
Oct 30 21:48:09.107923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.107813    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" event={"ID":"ed6323c9-8489-4ad1-86ed-0c588083ba26","Type":"ContainerDied","Data":"a1761962f7a8110950049d3b00744c699ad3cf1a022f1a92a8eda52f49634d46"}
Oct 30 21:48:09.109340 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.109313    5786 generic.go:356] "Generic (PLEG): container finished" podID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerID="eea172e9f44880ac1a70b06c92c382680f201647f98c8a3cc8d54469ad74836f" exitCode=0
Oct 30 21:48:09.109495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.109444    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" event={"ID":"89f6b2c2-83f6-4962-b811-4b5e5647d5bc","Type":"ContainerDied","Data":"eea172e9f44880ac1a70b06c92c382680f201647f98c8a3cc8d54469ad74836f"}
Oct 30 21:48:09.212437 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.212415    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:48:09.231782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.231761    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:48:09.320538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320513    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-dshm\") pod \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") "
Oct 30 21:48:09.320722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320549    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-kube-api-access-7mmgl\" (UniqueName: \"kubernetes.io/projected/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-kube-api-access-7mmgl\") pod \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") "
Oct 30 21:48:09.320722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320566    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-home\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-home\") pod \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") "
Oct 30 21:48:09.320722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320679    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed6323c9-8489-4ad1-86ed-0c588083ba26-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed6323c9-8489-4ad1-86ed-0c588083ba26-tls-certs\") pod \"ed6323c9-8489-4ad1-86ed-0c588083ba26\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") "
Oct 30 21:48:09.320894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320733    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-tmp-dir\") pod \"ed6323c9-8489-4ad1-86ed-0c588083ba26\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") "
Oct 30 21:48:09.320894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320865    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-home" pod "89f6b2c2-83f6-4962-b811-4b5e5647d5bc" (UID: "89f6b2c2-83f6-4962-b811-4b5e5647d5bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:09.320894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320872    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-home\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-home\") pod \"ed6323c9-8489-4ad1-86ed-0c588083ba26\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") "
Oct 30 21:48:09.321077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320921    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-dshm\") pod \"ed6323c9-8489-4ad1-86ed-0c588083ba26\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") "
Oct 30 21:48:09.321077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.320952    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tmp-dir\") pod \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") "
Oct 30 21:48:09.321077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321050    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tls-certs\" (UniqueName: \"kubernetes.io/secret/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tls-certs\") pod \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") "
Oct 30 21:48:09.321077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321050    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-tmp-dir" pod "ed6323c9-8489-4ad1-86ed-0c588083ba26" (UID: "ed6323c9-8489-4ad1-86ed-0c588083ba26"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:09.321346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321074    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-home" pod "ed6323c9-8489-4ad1-86ed-0c588083ba26" (UID: "ed6323c9-8489-4ad1-86ed-0c588083ba26"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:09.321346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321122    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed6323c9-8489-4ad1-86ed-0c588083ba26-kube-api-access-gbhfr\" (UniqueName: \"kubernetes.io/projected/ed6323c9-8489-4ad1-86ed-0c588083ba26-kube-api-access-gbhfr\") pod \"ed6323c9-8489-4ad1-86ed-0c588083ba26\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") "
Oct 30 21:48:09.321346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321265    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tmp-dir" pod "89f6b2c2-83f6-4962-b811-4b5e5647d5bc" (UID: "89f6b2c2-83f6-4962-b811-4b5e5647d5bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:09.321346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321273    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-model-cache\") pod \"ed6323c9-8489-4ad1-86ed-0c588083ba26\" (UID: \"ed6323c9-8489-4ad1-86ed-0c588083ba26\") "
Oct 30 21:48:09.321346 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321319    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-model-cache\") pod \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\" (UID: \"89f6b2c2-83f6-4962-b811-4b5e5647d5bc\") "
Oct 30 21:48:09.321707 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321684    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-model-cache" pod "ed6323c9-8489-4ad1-86ed-0c588083ba26" (UID: "ed6323c9-8489-4ad1-86ed-0c588083ba26"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:09.321988 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.321945    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-model-cache" pod "89f6b2c2-83f6-4962-b811-4b5e5647d5bc" (UID: "89f6b2c2-83f6-4962-b811-4b5e5647d5bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:09.322119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.322079    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.322119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.322102    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.322119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.322117    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.322326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.322131    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.322326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.322145    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.322326 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.322158    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.323515 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.323488    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-dshm" pod "ed6323c9-8489-4ad1-86ed-0c588083ba26" (UID: "ed6323c9-8489-4ad1-86ed-0c588083ba26"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:09.323643 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.323545    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed6323c9-8489-4ad1-86ed-0c588083ba26-tls-certs" pod "ed6323c9-8489-4ad1-86ed-0c588083ba26" (UID: "ed6323c9-8489-4ad1-86ed-0c588083ba26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:09.323699 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.323677    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-dshm" pod "89f6b2c2-83f6-4962-b811-4b5e5647d5bc" (UID: "89f6b2c2-83f6-4962-b811-4b5e5647d5bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:09.323827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.323808    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-kube-api-access-7mmgl" pod "89f6b2c2-83f6-4962-b811-4b5e5647d5bc" (UID: "89f6b2c2-83f6-4962-b811-4b5e5647d5bc"). InnerVolumeSpecName "kube-api-access-7mmgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:09.324178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.324159    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tls-certs" pod "89f6b2c2-83f6-4962-b811-4b5e5647d5bc" (UID: "89f6b2c2-83f6-4962-b811-4b5e5647d5bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:09.324352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.324332    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed6323c9-8489-4ad1-86ed-0c588083ba26-kube-api-access-gbhfr" pod "ed6323c9-8489-4ad1-86ed-0c588083ba26" (UID: "ed6323c9-8489-4ad1-86ed-0c588083ba26"). InnerVolumeSpecName "kube-api-access-gbhfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:09.423601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.423568    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.423743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.423662    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mmgl\" (UniqueName: \"kubernetes.io/projected/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-kube-api-access-7mmgl\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.423743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.423679    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed6323c9-8489-4ad1-86ed-0c588083ba26-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.423743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.423688    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed6323c9-8489-4ad1-86ed-0c588083ba26-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.423743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.423697    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89f6b2c2-83f6-4962-b811-4b5e5647d5bc-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:09.423743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:09.423714    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbhfr\" (UniqueName: \"kubernetes.io/projected/ed6323c9-8489-4ad1-86ed-0c588083ba26-kube-api-access-gbhfr\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:10.116720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.116697    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"
Oct 30 21:48:10.117120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.116695    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r" event={"ID":"ed6323c9-8489-4ad1-86ed-0c588083ba26","Type":"ContainerDied","Data":"2b51a92cd1a00debed34877cfb4c4559dd48d67444a7055d6cdfe315616b6a41"}
Oct 30 21:48:10.117120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.116803    5786 scope.go:122] "RemoveContainer" containerID="a1761962f7a8110950049d3b00744c699ad3cf1a022f1a92a8eda52f49634d46"
Oct 30 21:48:10.118236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.118192    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"
Oct 30 21:48:10.118236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.118227    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8" event={"ID":"89f6b2c2-83f6-4962-b811-4b5e5647d5bc","Type":"ContainerDied","Data":"ee71407a9c0fa95f0ae70c754f97da6164dd18aa422160c8e7ff4437fb98deb1"}
Oct 30 21:48:10.130388 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.130370    5786 scope.go:122] "RemoveContainer" containerID="eea172e9f44880ac1a70b06c92c382680f201647f98c8a3cc8d54469ad74836f"
Oct 30 21:48:10.277739 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.277691    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"]
Oct 30 21:48:10.326819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.326794    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/distinct-w2-38b832-kserve-5c5f7cd4ff-2wlr8"]
Oct 30 21:48:10.359188 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.359162    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" path="/var/lib/kubelet/pods/89f6b2c2-83f6-4962-b811-4b5e5647d5bc/volumes"
Oct 30 21:48:10.441659 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.441626    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"]
Oct 30 21:48:10.441793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:10.441680    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-38b832/simulator-w2-38b832-kserve-7766c47d5c-whp4r"]
Oct 30 21:48:12.359313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:12.359282    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" path="/var/lib/kubelet/pods/ed6323c9-8489-4ad1-86ed-0c588083ba26/volumes"
Oct 30 21:48:15.666627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:15.666589    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24" podUID="bb5f2a0f-9f81-4f88-8e65-bb4499139f87" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:48:22.353404 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:48:22.353366    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:48:25.079654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.079618    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"]
Oct 30 21:48:25.080339 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080322    5786 cpu_manager.go:414] "containerMap: removing container" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy"
Oct 30 21:48:25.080393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080340    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy"
Oct 30 21:48:25.080393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080352    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c6d5a236-8080-4cfb-a01e-0b4578d85c52" containerName="revoke-keys"
Oct 30 21:48:25.080393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080358    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6d5a236-8080-4cfb-a01e-0b4578d85c52" containerName="revoke-keys"
Oct 30 21:48:25.080393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080374    5786 cpu_manager.go:414] "containerMap: removing container" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy"
Oct 30 21:48:25.080393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080379    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy"
Oct 30 21:48:25.080393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080394    5786 cpu_manager.go:414] "containerMap: removing container" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main"
Oct 30 21:48:25.080582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080399    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main"
Oct 30 21:48:25.080582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080419    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main"
Oct 30 21:48:25.080582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080423    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main"
Oct 30 21:48:25.080582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080497    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="969f882f-d0de-48b8-839d-e33a4fd911fe" containerName="istio-proxy"
Oct 30 21:48:25.080582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080506    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6d5a236-8080-4cfb-a01e-0b4578d85c52" containerName="revoke-keys"
Oct 30 21:48:25.080582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080515    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6e7280e-7606-4185-913b-9142f7a88f56" containerName="istio-proxy"
Oct 30 21:48:25.080582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080522    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="89f6b2c2-83f6-4962-b811-4b5e5647d5bc" containerName="main"
Oct 30 21:48:25.080582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.080531    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed6323c9-8489-4ad1-86ed-0c588083ba26" containerName="main"
Oct 30 21:48:25.084079 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.084062    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.096289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.096268    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.096381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.096308    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2hhj\" (UniqueName: \"kubernetes.io/projected/b3eeb03c-de60-4d0f-af50-6217c7c02d48-kube-api-access-p2hhj\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.096475 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.096456    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.134915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.134888    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"]
Oct 30 21:48:25.139235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.139199    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.139363 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.139346    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"]
Oct 30 21:48:25.181236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.181197    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:48:25.186188 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.186167    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:48:25.196446 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.196422    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"]
Oct 30 21:48:25.197431 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.197012    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.197539 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.197489    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hhj\" (UniqueName: \"kubernetes.io/projected/b3eeb03c-de60-4d0f-af50-6217c7c02d48-kube-api-access-p2hhj\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.197630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.197602    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jhnx\" (UniqueName: \"kubernetes.io/projected/95a4b51d-7533-4db9-9928-d8dfca8752ee-kube-api-access-9jhnx\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.197855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.197652    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.197855 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.197803    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.197992 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.197864    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.198596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.198569    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.200076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.200053    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.253533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.253508    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2hhj\" (UniqueName: \"kubernetes.io/projected/b3eeb03c-de60-4d0f-af50-6217c7c02d48-kube-api-access-p2hhj\") pod \"maas-api-revoke-sub-8ceaeccc-m7ftx\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.298970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.298943    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.299097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.299079    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jhnx\" (UniqueName: \"kubernetes.io/projected/95a4b51d-7533-4db9-9928-d8dfca8752ee-kube-api-access-9jhnx\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.299168 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.299105    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.299630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.299609    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.301353 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.301335    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.355918 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.355865    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jhnx\" (UniqueName: \"kubernetes.io/projected/95a4b51d-7533-4db9-9928-d8dfca8752ee-kube-api-access-9jhnx\") pod \"maas-api-revoke-sub-f6d54aed-q6k8k\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.396907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.396886    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:25.451426 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.451386    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:25.667245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.667199    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24" podUID="bb5f2a0f-9f81-4f88-8e65-bb4499139f87" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:48:25.776399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.776375    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"]
Oct 30 21:48:25.777966 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:48:25.777945    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3eeb03c_de60_4d0f_af50_6217c7c02d48.slice/crio-14308aacc7146387166d5e2b1b6cf7c7f6815999c4a78be28d85ddc5f7b438f1 WatchSource:0}: Error finding container 14308aacc7146387166d5e2b1b6cf7c7f6815999c4a78be28d85ddc5f7b438f1: Status 404 returned error can't find the container with id 14308aacc7146387166d5e2b1b6cf7c7f6815999c4a78be28d85ddc5f7b438f1
Oct 30 21:48:25.841389 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:25.841371    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"]
Oct 30 21:48:25.841895 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:48:25.841873    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a4b51d_7533_4db9_9928_d8dfca8752ee.slice/crio-394b295e90435ebba4694350b307317b811ef19e678564adb0a4e972448807ea WatchSource:0}: Error finding container 394b295e90435ebba4694350b307317b811ef19e678564adb0a4e972448807ea: Status 404 returned error can't find the container with id 394b295e90435ebba4694350b307317b811ef19e678564adb0a4e972448807ea
Oct 30 21:48:26.186330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.186308    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:48:26.208659 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.208636    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-plugins-config-volume\") pod \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\" (UID: \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\") "
Oct 30 21:48:26.208771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.208758    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-kube-api-access-sth8c\" (UniqueName: \"kubernetes.io/projected/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-kube-api-access-sth8c\") pod \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\" (UID: \"bb5f2a0f-9f81-4f88-8e65-bb4499139f87\") "
Oct 30 21:48:26.208967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.208944    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-plugins-config-volume" pod "bb5f2a0f-9f81-4f88-8e65-bb4499139f87" (UID: "bb5f2a0f-9f81-4f88-8e65-bb4499139f87"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:26.210802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.209662    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:26.220848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.220822    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-kube-api-access-sth8c" pod "bb5f2a0f-9f81-4f88-8e65-bb4499139f87" (UID: "bb5f2a0f-9f81-4f88-8e65-bb4499139f87"). InnerVolumeSpecName "kube-api-access-sth8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:26.230295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.230266    5786 generic.go:356] "Generic (PLEG): container finished" podID="bb5f2a0f-9f81-4f88-8e65-bb4499139f87" containerID="f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e" exitCode=0
Oct 30 21:48:26.230391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.230318    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"
Oct 30 21:48:26.230391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.230344    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24" event={"ID":"bb5f2a0f-9f81-4f88-8e65-bb4499139f87","Type":"ContainerDied","Data":"f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e"}
Oct 30 21:48:26.230501 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.230421    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24" event={"ID":"bb5f2a0f-9f81-4f88-8e65-bb4499139f87","Type":"ContainerDied","Data":"8c28da515932a9f288f3411f70df93e492b3646cd43837246a2baecd0b777e91"}
Oct 30 21:48:26.230501 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.230448    5786 scope.go:122] "RemoveContainer" containerID="f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e"
Oct 30 21:48:26.232390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.232362    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx" event={"ID":"b3eeb03c-de60-4d0f-af50-6217c7c02d48","Type":"ContainerStarted","Data":"1f694bfaf61391f1f8436bc7b02692ec61e963752b31b52ca9d6580a8773bb81"}
Oct 30 21:48:26.232505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.232421    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx" event={"ID":"b3eeb03c-de60-4d0f-af50-6217c7c02d48","Type":"ContainerStarted","Data":"14308aacc7146387166d5e2b1b6cf7c7f6815999c4a78be28d85ddc5f7b438f1"}
Oct 30 21:48:26.233674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.233653    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k" event={"ID":"95a4b51d-7533-4db9-9928-d8dfca8752ee","Type":"ContainerStarted","Data":"394b295e90435ebba4694350b307317b811ef19e678564adb0a4e972448807ea"}
Oct 30 21:48:26.240788 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.240763    5786 scope.go:122] "RemoveContainer" containerID="f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e"
Oct 30 21:48:26.241046 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:48:26.241024    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e\": container with ID starting with f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e not found: ID does not exist" containerID="f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e"
Oct 30 21:48:26.241167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.241052    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e"} err="failed to get container status \"f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e\": rpc error: code = NotFound desc = could not find container \"f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e\": container with ID starting with f06921a6460abab878d07cd9a1eed7e415eb93bdd7fc11e20593c981cea3179e not found: ID does not exist"
Oct 30 21:48:26.311072 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.311042    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sth8c\" (UniqueName: \"kubernetes.io/projected/bb5f2a0f-9f81-4f88-8e65-bb4499139f87-kube-api-access-sth8c\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:26.410005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.409981    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"]
Oct 30 21:48:26.414028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.414006    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-38b832-fb6658bdc-vvh24"]
Oct 30 21:48:26.523791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.523706    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx" podStartSLOduration=1.523693234 podStartE2EDuration="1.523693234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:26.52094109 +0000 UTC m=+2162.392060209" watchObservedRunningTime="2026-10-30 21:48:26.523693234 +0000 UTC m=+2162.394812352"
Oct 30 21:48:26.953447 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.953417    5786 scope.go:122] "RemoveContainer" containerID="e7fcc7f3eed84a8247e62fbcc8fdea95782fd8cadcf305e51e138c8e4d68198e"
Oct 30 21:48:26.974928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:26.974904    5786 scope.go:122] "RemoveContainer" containerID="3062c5bc348f8375c5fb4a2457cec341690067fa6b428a168c7051fcf280f834"
Oct 30 21:48:27.052011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:27.051982    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"]
Oct 30 21:48:27.099599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:27.099547    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a0f3808-zkbks"]
Oct 30 21:48:27.241049 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:27.241018    5786 generic.go:356] "Generic (PLEG): container finished" podID="b3eeb03c-de60-4d0f-af50-6217c7c02d48" containerID="1f694bfaf61391f1f8436bc7b02692ec61e963752b31b52ca9d6580a8773bb81" exitCode=0
Oct 30 21:48:27.241499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:27.241105    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx" event={"ID":"b3eeb03c-de60-4d0f-af50-6217c7c02d48","Type":"ContainerDied","Data":"1f694bfaf61391f1f8436bc7b02692ec61e963752b31b52ca9d6580a8773bb81"}
Oct 30 21:48:27.242612 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:27.242592    5786 generic.go:356] "Generic (PLEG): container finished" podID="95a4b51d-7533-4db9-9928-d8dfca8752ee" containerID="bb2b3b4c1bdfe14b7c9549dcc136fe559eb4b5a8b341eab6bcca3bac57d3de97" exitCode=0
Oct 30 21:48:27.242712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:27.242666    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k" event={"ID":"95a4b51d-7533-4db9-9928-d8dfca8752ee","Type":"ContainerDied","Data":"bb2b3b4c1bdfe14b7c9549dcc136fe559eb4b5a8b341eab6bcca3bac57d3de97"}
Oct 30 21:48:28.049329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.049300    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"]
Oct 30 21:48:28.051893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.051868    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b89a9b36-7wcdl"]
Oct 30 21:48:28.360657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.360629    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2" path="/var/lib/kubelet/pods/9b71a1cd-1bb9-4ebf-bfd7-d603341c8ba2/volumes"
Oct 30 21:48:28.361083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.361067    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb5f2a0f-9f81-4f88-8e65-bb4499139f87" path="/var/lib/kubelet/pods/bb5f2a0f-9f81-4f88-8e65-bb4499139f87/volumes"
Oct 30 21:48:28.361519 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.361505    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec52b3ed-6a2a-4d0f-8e5b-639f54cac210" path="/var/lib/kubelet/pods/ec52b3ed-6a2a-4d0f-8e5b-639f54cac210/volumes"
Oct 30 21:48:28.422411 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.422391    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:28.424737 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.424719    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:28.549775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.549745    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3eeb03c-de60-4d0f-af50-6217c7c02d48-kube-api-access-p2hhj\" (UniqueName: \"kubernetes.io/projected/b3eeb03c-de60-4d0f-af50-6217c7c02d48-kube-api-access-p2hhj\") pod \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") "
Oct 30 21:48:28.549912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.549891    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-xks-ca\") pod \"95a4b51d-7533-4db9-9928-d8dfca8752ee\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") "
Oct 30 21:48:28.549975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.549958    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95a4b51d-7533-4db9-9928-d8dfca8752ee-kube-api-access-9jhnx\" (UniqueName: \"kubernetes.io/projected/95a4b51d-7533-4db9-9928-d8dfca8752ee-kube-api-access-9jhnx\") pod \"95a4b51d-7533-4db9-9928-d8dfca8752ee\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") "
Oct 30 21:48:28.550064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.550048    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-xks-ca\") pod \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") "
Oct 30 21:48:28.550145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.550113    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-openshift-ca\") pod \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\" (UID: \"b3eeb03c-de60-4d0f-af50-6217c7c02d48\") "
Oct 30 21:48:28.550231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.550159    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-openshift-ca\") pod \"95a4b51d-7533-4db9-9928-d8dfca8752ee\" (UID: \"95a4b51d-7533-4db9-9928-d8dfca8752ee\") "
Oct 30 21:48:28.551155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.550629    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-openshift-ca" pod "b3eeb03c-de60-4d0f-af50-6217c7c02d48" (UID: "b3eeb03c-de60-4d0f-af50-6217c7c02d48"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:28.551155 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.551120    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-openshift-ca" pod "95a4b51d-7533-4db9-9928-d8dfca8752ee" (UID: "95a4b51d-7533-4db9-9928-d8dfca8752ee"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:28.552532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.552503    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a4b51d-7533-4db9-9928-d8dfca8752ee-kube-api-access-9jhnx" pod "95a4b51d-7533-4db9-9928-d8dfca8752ee" (UID: "95a4b51d-7533-4db9-9928-d8dfca8752ee"). InnerVolumeSpecName "kube-api-access-9jhnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:28.553116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.553094    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-xks-ca" pod "95a4b51d-7533-4db9-9928-d8dfca8752ee" (UID: "95a4b51d-7533-4db9-9928-d8dfca8752ee"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:28.553116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.553108    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-xks-ca" pod "b3eeb03c-de60-4d0f-af50-6217c7c02d48" (UID: "b3eeb03c-de60-4d0f-af50-6217c7c02d48"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:28.553342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.553140    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3eeb03c-de60-4d0f-af50-6217c7c02d48-kube-api-access-p2hhj" pod "b3eeb03c-de60-4d0f-af50-6217c7c02d48" (UID: "b3eeb03c-de60-4d0f-af50-6217c7c02d48"). InnerVolumeSpecName "kube-api-access-p2hhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:28.651952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.651922    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jhnx\" (UniqueName: \"kubernetes.io/projected/95a4b51d-7533-4db9-9928-d8dfca8752ee-kube-api-access-9jhnx\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:28.652062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.651958    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:28.652062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.651973    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3eeb03c-de60-4d0f-af50-6217c7c02d48-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:28.652062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.651986    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:28.652062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.651999    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2hhj\" (UniqueName: \"kubernetes.io/projected/b3eeb03c-de60-4d0f-af50-6217c7c02d48-kube-api-access-p2hhj\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:28.652062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:28.652012    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a4b51d-7533-4db9-9928-d8dfca8752ee-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:29.257312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:29.257282    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"
Oct 30 21:48:29.257484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:29.257311    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx" event={"ID":"b3eeb03c-de60-4d0f-af50-6217c7c02d48","Type":"ContainerDied","Data":"14308aacc7146387166d5e2b1b6cf7c7f6815999c4a78be28d85ddc5f7b438f1"}
Oct 30 21:48:29.257484 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:29.257356    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14308aacc7146387166d5e2b1b6cf7c7f6815999c4a78be28d85ddc5f7b438f1"
Oct 30 21:48:29.258892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:29.258873    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"
Oct 30 21:48:29.258892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:29.258873    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k" event={"ID":"95a4b51d-7533-4db9-9928-d8dfca8752ee","Type":"ContainerDied","Data":"394b295e90435ebba4694350b307317b811ef19e678564adb0a4e972448807ea"}
Oct 30 21:48:29.259077 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:29.258916    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="394b295e90435ebba4694350b307317b811ef19e678564adb0a4e972448807ea"
Oct 30 21:48:52.320586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.320556    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"]
Oct 30 21:48:52.321124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321111    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b3eeb03c-de60-4d0f-af50-6217c7c02d48" containerName="revoke-keys"
Oct 30 21:48:52.321169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321125    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3eeb03c-de60-4d0f-af50-6217c7c02d48" containerName="revoke-keys"
Oct 30 21:48:52.321169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321137    5786 cpu_manager.go:414] "containerMap: removing container" podUID="bb5f2a0f-9f81-4f88-8e65-bb4499139f87" containerName="payload-processing"
Oct 30 21:48:52.321169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321142    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb5f2a0f-9f81-4f88-8e65-bb4499139f87" containerName="payload-processing"
Oct 30 21:48:52.321169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321163    5786 cpu_manager.go:414] "containerMap: removing container" podUID="95a4b51d-7533-4db9-9928-d8dfca8752ee" containerName="revoke-keys"
Oct 30 21:48:52.321169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321167    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95a4b51d-7533-4db9-9928-d8dfca8752ee" containerName="revoke-keys"
Oct 30 21:48:52.321406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321301    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="95a4b51d-7533-4db9-9928-d8dfca8752ee" containerName="revoke-keys"
Oct 30 21:48:52.321406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321319    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3eeb03c-de60-4d0f-af50-6217c7c02d48" containerName="revoke-keys"
Oct 30 21:48:52.321406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.321326    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb5f2a0f-9f81-4f88-8e65-bb4499139f87" containerName="payload-processing"
Oct 30 21:48:52.324758 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.324742    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.354368 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:48:52.354341    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:48:52.378405 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.378383    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:48:52.427521 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.427494    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:48:52.430192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.430173    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"]
Oct 30 21:48:52.502536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.502501    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.502701 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.502632    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.502814 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.502792    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnt8k\" (UniqueName: \"kubernetes.io/projected/a5f55486-de5a-45af-b41e-73032a90436a-kube-api-access-wnt8k\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.604188 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.604135    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnt8k\" (UniqueName: \"kubernetes.io/projected/a5f55486-de5a-45af-b41e-73032a90436a-kube-api-access-wnt8k\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.604308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.604267    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.604308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.604286    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.604826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.604802    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.606664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.606641    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.660809 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.660790    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnt8k\" (UniqueName: \"kubernetes.io/projected/a5f55486-de5a-45af-b41e-73032a90436a-kube-api-access-wnt8k\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:52.938492 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:52.938469    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:53.312480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:53.312449    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"]
Oct 30 21:48:53.314428 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:48:53.314392    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f55486_de5a_45af_b41e_73032a90436a.slice/crio-091bcd9202611831c04bff703d74aa3075fd83969ea5487479188713bc9a9209 WatchSource:0}: Error finding container 091bcd9202611831c04bff703d74aa3075fd83969ea5487479188713bc9a9209: Status 404 returned error can't find the container with id 091bcd9202611831c04bff703d74aa3075fd83969ea5487479188713bc9a9209
Oct 30 21:48:53.417977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:53.417941    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr" event={"ID":"a5f55486-de5a-45af-b41e-73032a90436a","Type":"ContainerStarted","Data":"091bcd9202611831c04bff703d74aa3075fd83969ea5487479188713bc9a9209"}
Oct 30 21:48:54.426597 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:54.426569    5786 generic.go:356] "Generic (PLEG): container finished" podID="a5f55486-de5a-45af-b41e-73032a90436a" containerID="3564d67ca0aa65681705c7401ad57fca0278684e21b7f40837c660de79a4e401" exitCode=0
Oct 30 21:48:54.426963 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:54.426622    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr" event={"ID":"a5f55486-de5a-45af-b41e-73032a90436a","Type":"ContainerDied","Data":"3564d67ca0aa65681705c7401ad57fca0278684e21b7f40837c660de79a4e401"}
Oct 30 21:48:55.566216 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.566182    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:55.637772 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.637752    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-xks-ca\") pod \"a5f55486-de5a-45af-b41e-73032a90436a\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") "
Oct 30 21:48:55.637932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.637915    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-openshift-ca\") pod \"a5f55486-de5a-45af-b41e-73032a90436a\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") "
Oct 30 21:48:55.638027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.638012    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5f55486-de5a-45af-b41e-73032a90436a-kube-api-access-wnt8k\" (UniqueName: \"kubernetes.io/projected/a5f55486-de5a-45af-b41e-73032a90436a-kube-api-access-wnt8k\") pod \"a5f55486-de5a-45af-b41e-73032a90436a\" (UID: \"a5f55486-de5a-45af-b41e-73032a90436a\") "
Oct 30 21:48:55.638247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.638226    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-openshift-ca" pod "a5f55486-de5a-45af-b41e-73032a90436a" (UID: "a5f55486-de5a-45af-b41e-73032a90436a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:55.638499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.638446    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:55.639898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.639874    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-xks-ca" pod "a5f55486-de5a-45af-b41e-73032a90436a" (UID: "a5f55486-de5a-45af-b41e-73032a90436a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:55.640067 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.640051    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f55486-de5a-45af-b41e-73032a90436a-kube-api-access-wnt8k" pod "a5f55486-de5a-45af-b41e-73032a90436a" (UID: "a5f55486-de5a-45af-b41e-73032a90436a"). InnerVolumeSpecName "kube-api-access-wnt8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:55.739682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.739624    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnt8k\" (UniqueName: \"kubernetes.io/projected/a5f55486-de5a-45af-b41e-73032a90436a-kube-api-access-wnt8k\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:55.739682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:55.739648    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5f55486-de5a-45af-b41e-73032a90436a-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:56.443163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:56.443125    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"
Oct 30 21:48:56.443343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:56.443163    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr" event={"ID":"a5f55486-de5a-45af-b41e-73032a90436a","Type":"ContainerDied","Data":"091bcd9202611831c04bff703d74aa3075fd83969ea5487479188713bc9a9209"}
Oct 30 21:48:56.443343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:48:56.443240    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="091bcd9202611831c04bff703d74aa3075fd83969ea5487479188713bc9a9209"
Oct 30 21:49:03.833635 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:03.833604    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"]
Oct 30 21:49:03.937225 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:03.937188    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"]
Oct 30 21:49:08.834937 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:08.834899    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7" podUID="b0116a4e-e09d-4d5a-9948-369d7cbb17ea" containerName="payload-processing" containerID="cri-o://2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9" gracePeriod=25
Oct 30 21:49:08.937716 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:08.937656    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt" podUID="e6b9c5cd-2e9b-4cec-8499-17c84cd36401" containerName="payload-pre-processing" containerID="cri-o://408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a" gracePeriod=25
Oct 30 21:49:09.113943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.113914    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:49:09.187851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.187819    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-kube-api-access-kjnx6\" (UniqueName: \"kubernetes.io/projected/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-kube-api-access-kjnx6\") pod \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\" (UID: \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\") "
Oct 30 21:49:09.188001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.187933    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-plugins-config-volume\") pod \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\" (UID: \"b0116a4e-e09d-4d5a-9948-369d7cbb17ea\") "
Oct 30 21:49:09.188332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.188305    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-plugins-config-volume" pod "b0116a4e-e09d-4d5a-9948-369d7cbb17ea" (UID: "b0116a4e-e09d-4d5a-9948-369d7cbb17ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:09.189912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.189891    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-kube-api-access-kjnx6" pod "b0116a4e-e09d-4d5a-9948-369d7cbb17ea" (UID: "b0116a4e-e09d-4d5a-9948-369d7cbb17ea"). InnerVolumeSpecName "kube-api-access-kjnx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:09.206761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.206741    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:49:09.288940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.288914    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-plugins-config-volume\") pod \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\" (UID: \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\") "
Oct 30 21:49:09.289103 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.289034    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-kube-api-access-lggls\" (UniqueName: \"kubernetes.io/projected/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-kube-api-access-lggls\") pod \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\" (UID: \"e6b9c5cd-2e9b-4cec-8499-17c84cd36401\") "
Oct 30 21:49:09.289304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.289282    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-plugins-config-volume" pod "e6b9c5cd-2e9b-4cec-8499-17c84cd36401" (UID: "e6b9c5cd-2e9b-4cec-8499-17c84cd36401"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:09.289500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.289487    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:09.289537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.289503    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjnx6\" (UniqueName: \"kubernetes.io/projected/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-kube-api-access-kjnx6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:09.289537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.289513    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0116a4e-e09d-4d5a-9948-369d7cbb17ea-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:09.291144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.291126    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-kube-api-access-lggls" pod "e6b9c5cd-2e9b-4cec-8499-17c84cd36401" (UID: "e6b9c5cd-2e9b-4cec-8499-17c84cd36401"). InnerVolumeSpecName "kube-api-access-lggls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:09.391024 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.390955    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lggls\" (UniqueName: \"kubernetes.io/projected/e6b9c5cd-2e9b-4cec-8499-17c84cd36401-kube-api-access-lggls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:09.534638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.534605    5786 generic.go:356] "Generic (PLEG): container finished" podID="e6b9c5cd-2e9b-4cec-8499-17c84cd36401" containerID="408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a" exitCode=0
Oct 30 21:49:09.534835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.534649    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"
Oct 30 21:49:09.534835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.534678    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt" event={"ID":"e6b9c5cd-2e9b-4cec-8499-17c84cd36401","Type":"ContainerDied","Data":"408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a"}
Oct 30 21:49:09.534835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.534735    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt" event={"ID":"e6b9c5cd-2e9b-4cec-8499-17c84cd36401","Type":"ContainerDied","Data":"9cb990ce8b1687c6a54da98d2fc946a9472bf3d2cfdf783156f0bd2ae81ad188"}
Oct 30 21:49:09.534835 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.534769    5786 scope.go:122] "RemoveContainer" containerID="408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a"
Oct 30 21:49:09.536090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.536072    5786 generic.go:356] "Generic (PLEG): container finished" podID="b0116a4e-e09d-4d5a-9948-369d7cbb17ea" containerID="2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9" exitCode=0
Oct 30 21:49:09.536197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.536115    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"
Oct 30 21:49:09.536283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.536117    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7" event={"ID":"b0116a4e-e09d-4d5a-9948-369d7cbb17ea","Type":"ContainerDied","Data":"2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9"}
Oct 30 21:49:09.536283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.536261    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7" event={"ID":"b0116a4e-e09d-4d5a-9948-369d7cbb17ea","Type":"ContainerDied","Data":"6078b0d206168f3741efe2b5fe7dca47fdd6ea23b2e517d9cd1abdda30c971b0"}
Oct 30 21:49:09.546145 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.546126    5786 scope.go:122] "RemoveContainer" containerID="408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a"
Oct 30 21:49:09.546429 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:09.546408    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a\": container with ID starting with 408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a not found: ID does not exist" containerID="408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a"
Oct 30 21:49:09.546506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.546441    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a"} err="failed to get container status \"408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a\": rpc error: code = NotFound desc = could not find container \"408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a\": container with ID starting with 408741f45c38a20f409eb09ee5437cd68aee0ec29b5499452a081fa9de637e3a not found: ID does not exist"
Oct 30 21:49:09.546506 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.546464    5786 scope.go:122] "RemoveContainer" containerID="2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9"
Oct 30 21:49:09.556200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.556184    5786 scope.go:122] "RemoveContainer" containerID="2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9"
Oct 30 21:49:09.556509 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:09.556490    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9\": container with ID starting with 2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9 not found: ID does not exist" containerID="2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9"
Oct 30 21:49:09.556573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.556518    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9"} err="failed to get container status \"2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9\": rpc error: code = NotFound desc = could not find container \"2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9\": container with ID starting with 2b025a9c20946523a33676f528e006f7f29db87b1b2bf3d625c2f333247e60d9 not found: ID does not exist"
Oct 30 21:49:09.573867 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:09.573844    5786 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0116a4e_e09d_4d5a_9948_369d7cbb17ea.slice/crio-6078b0d206168f3741efe2b5fe7dca47fdd6ea23b2e517d9cd1abdda30c971b0\": RecentStats: unable to find data in memory cache]"
Oct 30 21:49:09.696769 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.696739    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"]
Oct 30 21:49:09.702341 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.702314    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fe5385-7494947467-w86kt"]
Oct 30 21:49:09.808784 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.808760    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"]
Oct 30 21:49:09.811815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:09.811793    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-fe5385-594db586d9-tdzr7"]
Oct 30 21:49:10.014361 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:10.014280    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"]
Oct 30 21:49:10.016694 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:10.016670    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-38b832-6pjfj"]
Oct 30 21:49:10.359857 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:10.359787    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0116a4e-e09d-4d5a-9948-369d7cbb17ea" path="/var/lib/kubelet/pods/b0116a4e-e09d-4d5a-9948-369d7cbb17ea/volumes"
Oct 30 21:49:10.360192 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:10.360178    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6d5a236-8080-4cfb-a01e-0b4578d85c52" path="/var/lib/kubelet/pods/c6d5a236-8080-4cfb-a01e-0b4578d85c52/volumes"
Oct 30 21:49:10.360543 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:10.360530    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6b9c5cd-2e9b-4cec-8499-17c84cd36401" path="/var/lib/kubelet/pods/e6b9c5cd-2e9b-4cec-8499-17c84cd36401/volumes"
Oct 30 21:49:12.779284 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779253    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"]
Oct 30 21:49:12.779802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779789    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a5f55486-de5a-45af-b41e-73032a90436a" containerName="revoke-keys"
Oct 30 21:49:12.779851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779803    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5f55486-de5a-45af-b41e-73032a90436a" containerName="revoke-keys"
Oct 30 21:49:12.779851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779818    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b0116a4e-e09d-4d5a-9948-369d7cbb17ea" containerName="payload-processing"
Oct 30 21:49:12.779851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779823    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0116a4e-e09d-4d5a-9948-369d7cbb17ea" containerName="payload-processing"
Oct 30 21:49:12.779851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779833    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e6b9c5cd-2e9b-4cec-8499-17c84cd36401" containerName="payload-pre-processing"
Oct 30 21:49:12.779851 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779838    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6b9c5cd-2e9b-4cec-8499-17c84cd36401" containerName="payload-pre-processing"
Oct 30 21:49:12.779997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779925    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0116a4e-e09d-4d5a-9948-369d7cbb17ea" containerName="payload-processing"
Oct 30 21:49:12.779997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779932    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6b9c5cd-2e9b-4cec-8499-17c84cd36401" containerName="payload-pre-processing"
Oct 30 21:49:12.779997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.779947    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5f55486-de5a-45af-b41e-73032a90436a" containerName="revoke-keys"
Oct 30 21:49:12.783090 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.783076    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:12.885303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.885272    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:49:12.885303 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.885275    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:49:12.887913 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.887893    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"]
Oct 30 21:49:12.925007 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.924981    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:12.925128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.925011    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:12.925128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:12.925041    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9mpg\" (UniqueName: \"kubernetes.io/projected/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-kube-api-access-c9mpg\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:13.025939 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.025916    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:13.026076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.025945    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:13.026076 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.025975    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9mpg\" (UniqueName: \"kubernetes.io/projected/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-kube-api-access-c9mpg\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:13.026582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.026565    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:13.028396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.028369    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:13.081764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.081710    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9mpg\" (UniqueName: \"kubernetes.io/projected/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-kube-api-access-c9mpg\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:13.096793 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.096768    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:13.469404 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.469381    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"]
Oct 30 21:49:13.471314 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:49:13.471286    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfc1aa2_ba05_4f20_a37c_bbd2b466ff5a.slice/crio-bf7ce1bcd384fbd349a211b357c665f0d313b99ea148c608e95c6a4b05647370 WatchSource:0}: Error finding container bf7ce1bcd384fbd349a211b357c665f0d313b99ea148c608e95c6a4b05647370: Status 404 returned error can't find the container with id bf7ce1bcd384fbd349a211b357c665f0d313b99ea148c608e95c6a4b05647370
Oct 30 21:49:13.563520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:13.563493    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4" event={"ID":"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a","Type":"ContainerStarted","Data":"bf7ce1bcd384fbd349a211b357c665f0d313b99ea148c608e95c6a4b05647370"}
Oct 30 21:49:14.571517 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:14.571445    5786 generic.go:356] "Generic (PLEG): container finished" podID="bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a" containerID="63355943e1b0de3ca3ed88c615d8fb8fa2472d85d4a649330f8aced523915078" exitCode=0
Oct 30 21:49:14.571891 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:14.571527    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4" event={"ID":"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a","Type":"ContainerDied","Data":"63355943e1b0de3ca3ed88c615d8fb8fa2472d85d4a649330f8aced523915078"}
Oct 30 21:49:15.712356 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.712332    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:15.860827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.860762    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-kube-api-access-c9mpg\" (UniqueName: \"kubernetes.io/projected/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-kube-api-access-c9mpg\") pod \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") "
Oct 30 21:49:15.861013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.860926    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-openshift-ca\") pod \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") "
Oct 30 21:49:15.861013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.860955    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-xks-ca\") pod \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\" (UID: \"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a\") "
Oct 30 21:49:15.861335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.861309    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-openshift-ca" pod "bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a" (UID: "bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:15.862954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.862924    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-kube-api-access-c9mpg" pod "bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a" (UID: "bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a"). InnerVolumeSpecName "kube-api-access-c9mpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:15.863039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.863014    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-xks-ca" pod "bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a" (UID: "bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:15.963036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.962996    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:15.963036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.963026    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:15.963036 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:15.963036    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9mpg\" (UniqueName: \"kubernetes.io/projected/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a-kube-api-access-c9mpg\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:16.585604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:16.585572    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"
Oct 30 21:49:16.585762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:16.585613    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4" event={"ID":"bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a","Type":"ContainerDied","Data":"bf7ce1bcd384fbd349a211b357c665f0d313b99ea148c608e95c6a4b05647370"}
Oct 30 21:49:16.585762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:16.585657    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf7ce1bcd384fbd349a211b357c665f0d313b99ea148c608e95c6a4b05647370"
Oct 30 21:49:22.354404 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:22.354371    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:49:24.277285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:24.277257    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"]
Oct 30 21:49:24.382890 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:24.382862    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"]
Oct 30 21:49:25.915600 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:25.915571    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"]
Oct 30 21:49:25.970108 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:25.970074    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-fe5385-9kjxr"]
Oct 30 21:49:26.358762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:26.358674    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5f55486-de5a-45af-b41e-73032a90436a" path="/var/lib/kubelet/pods/a5f55486-de5a-45af-b41e-73032a90436a/volumes"
Oct 30 21:49:27.032890 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:27.032855    5786 scope.go:122] "RemoveContainer" containerID="4f9a607fb5da8fa7caaa64e1589a000fe5cb6967c33eb6da8d3d4fd90d6a229f"
Oct 30 21:49:27.043947 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:27.043930    5786 scope.go:122] "RemoveContainer" containerID="da6c6ef444aa2cd4f73ecea1a91a79dc7117c0e04c19316e39823c4dd74be259"
Oct 30 21:49:27.053309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:27.053290    5786 scope.go:122] "RemoveContainer" containerID="be8b7ec82be7e3ddb4dab3bff8a8ea565bb6321ca7d7c24effcf7a6605b2925b"
Oct 30 21:49:27.061847 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:27.061825    5786 scope.go:122] "RemoveContainer" containerID="56a4115114e811792d766a0fd6cd95dff6b0f2976f8a1aa657c6a8877be58ac0"
Oct 30 21:49:27.070171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:27.070155    5786 scope.go:122] "RemoveContainer" containerID="a22b96f79e09eb9596e20b192836b3f0da2cc8b56784a4683f712546dd8db282"
Oct 30 21:49:29.278055 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:29.278004    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58" podUID="00878b67-261a-4b77-ac91-a29de49126ae" containerName="payload-processing" containerID="cri-o://d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6" gracePeriod=25
Oct 30 21:49:29.383289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:29.383240    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b" podUID="903ce8c0-dd05-491c-a34f-344fd7275b31" containerName="payload-pre-processing" containerID="cri-o://c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234" gracePeriod=25
Oct 30 21:49:30.100370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:30.100338    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58" podUID="00878b67-261a-4b77-ac91-a29de49126ae" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:30.100370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:30.100351    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b" podUID="903ce8c0-dd05-491c-a34f-344fd7275b31" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:33.311276 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:33.311243    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"]
Oct 30 21:49:33.311716 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:33.311441    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy" containerID="cri-o://8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce" gracePeriod=30
Oct 30 21:49:34.067282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.067255    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"]
Oct 30 21:49:34.067930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.067912    5786 cpu_manager.go:414] "containerMap: removing container" podUID="bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a" containerName="revoke-keys"
Oct 30 21:49:34.067995 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.067932    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a" containerName="revoke-keys"
Oct 30 21:49:34.068042 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.068032    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a" containerName="revoke-keys"
Oct 30 21:49:34.073169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.073149    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.129974 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.129942    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"]
Oct 30 21:49:34.175428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.175405    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:49:34.175566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.175437    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:49:34.227465 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.227434    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.227595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.227488    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fkts\" (UniqueName: \"kubernetes.io/projected/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-kube-api-access-6fkts\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.227638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.227616    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.328381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.328321    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fkts\" (UniqueName: \"kubernetes.io/projected/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-kube-api-access-6fkts\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.328690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.328391    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.328690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.328471    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.329002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.328981    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.330858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.330838    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.384123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.384099    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fkts\" (UniqueName: \"kubernetes.io/projected/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-kube-api-access-6fkts\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.386936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.386917    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:34.762082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:34.762059    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"]
Oct 30 21:49:34.764359 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:49:34.764332    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73990eb0_b383_4131_85e5_5dd7b6bdf6a5.slice/crio-e7fa1059ad30a63281116840ea894d94ca88b214ba3d0950a7507e4b2c763f2d WatchSource:0}: Error finding container e7fa1059ad30a63281116840ea894d94ca88b214ba3d0950a7507e4b2c763f2d: Status 404 returned error can't find the container with id e7fa1059ad30a63281116840ea894d94ca88b214ba3d0950a7507e4b2c763f2d
Oct 30 21:49:35.737558 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:35.737526    5786 generic.go:356] "Generic (PLEG): container finished" podID="73990eb0-b383-4131-85e5-5dd7b6bdf6a5" containerID="8fdf6141854ee31d7506ddc79e88c767d8eb4977b0e9be20b0e6903a7475fd2b" exitCode=0
Oct 30 21:49:35.737984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:35.737612    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29" event={"ID":"73990eb0-b383-4131-85e5-5dd7b6bdf6a5","Type":"ContainerDied","Data":"8fdf6141854ee31d7506ddc79e88c767d8eb4977b0e9be20b0e6903a7475fd2b"}
Oct 30 21:49:35.737984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:35.737671    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29" event={"ID":"73990eb0-b383-4131-85e5-5dd7b6bdf6a5","Type":"ContainerStarted","Data":"e7fa1059ad30a63281116840ea894d94ca88b214ba3d0950a7507e4b2c763f2d"}
Oct 30 21:49:36.889086 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:36.889065    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:37.058811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.058745    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-kube-api-access-6fkts\" (UniqueName: \"kubernetes.io/projected/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-kube-api-access-6fkts\") pod \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") "
Oct 30 21:49:37.058931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.058917    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-xks-ca\") pod \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") "
Oct 30 21:49:37.058976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.058960    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-openshift-ca\") pod \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\" (UID: \"73990eb0-b383-4131-85e5-5dd7b6bdf6a5\") "
Oct 30 21:49:37.059358 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.059333    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-openshift-ca" pod "73990eb0-b383-4131-85e5-5dd7b6bdf6a5" (UID: "73990eb0-b383-4131-85e5-5dd7b6bdf6a5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:37.059489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.059461    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:37.060869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.060847    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-kube-api-access-6fkts" pod "73990eb0-b383-4131-85e5-5dd7b6bdf6a5" (UID: "73990eb0-b383-4131-85e5-5dd7b6bdf6a5"). InnerVolumeSpecName "kube-api-access-6fkts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:37.061152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.061134    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-xks-ca" pod "73990eb0-b383-4131-85e5-5dd7b6bdf6a5" (UID: "73990eb0-b383-4131-85e5-5dd7b6bdf6a5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:37.160817 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.160790    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:37.160935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.160815    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fkts\" (UniqueName: \"kubernetes.io/projected/73990eb0-b383-4131-85e5-5dd7b6bdf6a5-kube-api-access-6fkts\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:37.752958 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.752931    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"
Oct 30 21:49:37.753119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.752966    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29" event={"ID":"73990eb0-b383-4131-85e5-5dd7b6bdf6a5","Type":"ContainerDied","Data":"e7fa1059ad30a63281116840ea894d94ca88b214ba3d0950a7507e4b2c763f2d"}
Oct 30 21:49:37.753119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:37.753007    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7fa1059ad30a63281116840ea894d94ca88b214ba3d0950a7507e4b2c763f2d"
Oct 30 21:49:38.573106 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.573054    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:49:38.676330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676293    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-socket\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676371    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-certs\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676419    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-kube-api-access-gm8nb\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-kube-api-access-gm8nb\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676547 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676464    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-data\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676547 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676505    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-credential-socket\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676547 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676514    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-socket" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:38.676657 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676560    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa8f6777-2d13-4940-b11a-4083c144496d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa8f6777-2d13-4940-b11a-4083c144496d-istiod-ca-cert\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676693 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676652    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa8f6777-2d13-4940-b11a-4083c144496d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa8f6777-2d13-4940-b11a-4083c144496d-istio-podinfo\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676737 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676691    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-envoy\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676737 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676693    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-certs" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:38.676825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676739    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-credential-socket" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:38.676825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676760    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-istio-token\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-istio-token\") pod \"aa8f6777-2d13-4940-b11a-4083c144496d\" (UID: \"aa8f6777-2d13-4940-b11a-4083c144496d\") "
Oct 30 21:49:38.676825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676753    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-data" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:38.677043 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.676902    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa8f6777-2d13-4940-b11a-4083c144496d-istiod-ca-cert" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:38.677189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.677171    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.677290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.677191    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.677290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.677201    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.677290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.677241    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.677290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.677253    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa8f6777-2d13-4940-b11a-4083c144496d-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.679151 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.679123    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-istio-token" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:38.679340 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.679324    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-envoy" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:38.679409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.679342    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa8f6777-2d13-4940-b11a-4083c144496d-istio-podinfo" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:38.679409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.679380    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-kube-api-access-gm8nb" pod "aa8f6777-2d13-4940-b11a-4083c144496d" (UID: "aa8f6777-2d13-4940-b11a-4083c144496d"). InnerVolumeSpecName "kube-api-access-gm8nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:38.760991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.760965    5786 generic.go:356] "Generic (PLEG): container finished" podID="aa8f6777-2d13-4940-b11a-4083c144496d" containerID="8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce" exitCode=0
Oct 30 21:49:38.761134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.761010    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"
Oct 30 21:49:38.761134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.761028    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" event={"ID":"aa8f6777-2d13-4940-b11a-4083c144496d","Type":"ContainerDied","Data":"8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce"}
Oct 30 21:49:38.761134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.761081    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz" event={"ID":"aa8f6777-2d13-4940-b11a-4083c144496d","Type":"ContainerDied","Data":"4fff7f14748b0d35e4938dcca9357efb5f26c73effad4911bd2f737e1f4d1bf1"}
Oct 30 21:49:38.761134 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.761109    5786 scope.go:122] "RemoveContainer" containerID="8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce"
Oct 30 21:49:38.772284 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.772263    5786 scope.go:122] "RemoveContainer" containerID="efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995"
Oct 30 21:49:38.779025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.779000    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm8nb\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-kube-api-access-gm8nb\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.779107 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.779034    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa8f6777-2d13-4940-b11a-4083c144496d-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.779107 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.779050    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa8f6777-2d13-4940-b11a-4083c144496d-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.779107 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.779063    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa8f6777-2d13-4940-b11a-4083c144496d-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:38.783252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.783231    5786 scope.go:122] "RemoveContainer" containerID="8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce"
Oct 30 21:49:38.783538 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:38.783520    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce\": container with ID starting with 8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce not found: ID does not exist" containerID="8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce"
Oct 30 21:49:38.783586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.783546    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce"} err="failed to get container status \"8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce\": rpc error: code = NotFound desc = could not find container \"8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce\": container with ID starting with 8860484f94387a7ec4994e1bf725ec5fb974e538ff3e27f50b5f57abb114a8ce not found: ID does not exist"
Oct 30 21:49:38.783586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.783564    5786 scope.go:122] "RemoveContainer" containerID="efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995"
Oct 30 21:49:38.783846 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:38.783828    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995\": container with ID starting with efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995 not found: ID does not exist" containerID="efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995"
Oct 30 21:49:38.783897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.783852    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995"} err="failed to get container status \"efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995\": rpc error: code = NotFound desc = could not find container \"efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995\": container with ID starting with efe1ab294057ffa06353000a45b3a801c65fb50181856a7bfd9df90b1afe4995 not found: ID does not exist"
Oct 30 21:49:38.924334 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.924302    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"]
Oct 30 21:49:38.927782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:38.927760    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-fe5385-openshift-default-7c54856b75-w2jxz"]
Oct 30 21:49:40.099691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:40.099658    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58" podUID="00878b67-261a-4b77-ac91-a29de49126ae" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:40.099691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:40.099670    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b" podUID="903ce8c0-dd05-491c-a34f-344fd7275b31" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:40.358712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:40.358639    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" path="/var/lib/kubelet/pods/aa8f6777-2d13-4940-b11a-4083c144496d/volumes"
Oct 30 21:49:44.614343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.614305    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"]
Oct 30 21:49:44.615035 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615012    5786 cpu_manager.go:414] "containerMap: removing container" podUID="73990eb0-b383-4131-85e5-5dd7b6bdf6a5" containerName="revoke-keys"
Oct 30 21:49:44.615142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615036    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73990eb0-b383-4131-85e5-5dd7b6bdf6a5" containerName="revoke-keys"
Oct 30 21:49:44.615142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615061    5786 cpu_manager.go:414] "containerMap: removing container" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy"
Oct 30 21:49:44.615142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615069    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy"
Oct 30 21:49:44.615142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615092    5786 cpu_manager.go:414] "containerMap: removing container" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy"
Oct 30 21:49:44.615142 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615099    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy"
Oct 30 21:49:44.615459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615258    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="73990eb0-b383-4131-85e5-5dd7b6bdf6a5" containerName="revoke-keys"
Oct 30 21:49:44.615459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615276    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy"
Oct 30 21:49:44.615459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.615289    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa8f6777-2d13-4940-b11a-4083c144496d" containerName="istio-proxy"
Oct 30 21:49:44.618661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.618639    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.676285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.676261    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"]
Oct 30 21:49:44.721691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.721665    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:49:44.721799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.721703    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:49:44.737249 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.737226    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.737370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.737257    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.737419 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.737385    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq4d8\" (UniqueName: \"kubernetes.io/projected/568d4be3-420b-4b37-af8b-51474561b8dc-kube-api-access-zq4d8\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.838740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.838716    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.838872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.838755    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.838872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.838852    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq4d8\" (UniqueName: \"kubernetes.io/projected/568d4be3-420b-4b37-af8b-51474561b8dc-kube-api-access-zq4d8\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.839454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.839435    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.840972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.840952    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.894832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.894779    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq4d8\" (UniqueName: \"kubernetes.io/projected/568d4be3-420b-4b37-af8b-51474561b8dc-kube-api-access-zq4d8\") pod \"maas-api-revoke-sub-4b82c593-98bld\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:44.931312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:44.931292    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:45.313862 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:45.313824    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"]
Oct 30 21:49:45.315765 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:49:45.315740    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568d4be3_420b_4b37_af8b_51474561b8dc.slice/crio-8f339f5ae44f471c7abb2599296584ab7a73c807bafa55dbade527725ac6c4c4 WatchSource:0}: Error finding container 8f339f5ae44f471c7abb2599296584ab7a73c807bafa55dbade527725ac6c4c4: Status 404 returned error can't find the container with id 8f339f5ae44f471c7abb2599296584ab7a73c807bafa55dbade527725ac6c4c4
Oct 30 21:49:45.544648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:45.544613    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"]
Oct 30 21:49:45.654803 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:45.652970    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"]
Oct 30 21:49:45.805540 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:45.805508    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld" event={"ID":"568d4be3-420b-4b37-af8b-51474561b8dc","Type":"ContainerStarted","Data":"6d68bac060ba6a232581f4765262382dcaa3dee8e3e282a534b5c5f6ba58a2c9"}
Oct 30 21:49:45.805714 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:45.805556    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld" event={"ID":"568d4be3-420b-4b37-af8b-51474561b8dc","Type":"ContainerStarted","Data":"8f339f5ae44f471c7abb2599296584ab7a73c807bafa55dbade527725ac6c4c4"}
Oct 30 21:49:45.920634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:45.920587    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld" podStartSLOduration=1.920572546 podStartE2EDuration="1.920572546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:45.918858327 +0000 UTC m=+2241.789977445" watchObservedRunningTime="2026-10-30 21:49:45.920572546 +0000 UTC m=+2241.791691664"
Oct 30 21:49:46.812946 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:46.812918    5786 generic.go:356] "Generic (PLEG): container finished" podID="568d4be3-420b-4b37-af8b-51474561b8dc" containerID="6d68bac060ba6a232581f4765262382dcaa3dee8e3e282a534b5c5f6ba58a2c9" exitCode=0
Oct 30 21:49:46.813360 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:46.812989    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld" event={"ID":"568d4be3-420b-4b37-af8b-51474561b8dc","Type":"ContainerDied","Data":"6d68bac060ba6a232581f4765262382dcaa3dee8e3e282a534b5c5f6ba58a2c9"}
Oct 30 21:49:47.964601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:47.964579    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:48.010967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.010942    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-openshift-ca\") pod \"568d4be3-420b-4b37-af8b-51474561b8dc\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") "
Oct 30 21:49:48.011114 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.011096    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/568d4be3-420b-4b37-af8b-51474561b8dc-kube-api-access-zq4d8\" (UniqueName: \"kubernetes.io/projected/568d4be3-420b-4b37-af8b-51474561b8dc-kube-api-access-zq4d8\") pod \"568d4be3-420b-4b37-af8b-51474561b8dc\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") "
Oct 30 21:49:48.011183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.011168    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-xks-ca\") pod \"568d4be3-420b-4b37-af8b-51474561b8dc\" (UID: \"568d4be3-420b-4b37-af8b-51474561b8dc\") "
Oct 30 21:49:48.011350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.011330    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-openshift-ca" pod "568d4be3-420b-4b37-af8b-51474561b8dc" (UID: "568d4be3-420b-4b37-af8b-51474561b8dc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:48.011599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.011578    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:48.013242 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.013198    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-xks-ca" pod "568d4be3-420b-4b37-af8b-51474561b8dc" (UID: "568d4be3-420b-4b37-af8b-51474561b8dc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:48.013317 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.013237    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568d4be3-420b-4b37-af8b-51474561b8dc-kube-api-access-zq4d8" pod "568d4be3-420b-4b37-af8b-51474561b8dc" (UID: "568d4be3-420b-4b37-af8b-51474561b8dc"). InnerVolumeSpecName "kube-api-access-zq4d8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:48.112491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.112432    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/568d4be3-420b-4b37-af8b-51474561b8dc-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:48.112491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.112460    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq4d8\" (UniqueName: \"kubernetes.io/projected/568d4be3-420b-4b37-af8b-51474561b8dc-kube-api-access-zq4d8\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:48.826037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.826004    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld" event={"ID":"568d4be3-420b-4b37-af8b-51474561b8dc","Type":"ContainerDied","Data":"8f339f5ae44f471c7abb2599296584ab7a73c807bafa55dbade527725ac6c4c4"}
Oct 30 21:49:48.826037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.826038    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"
Oct 30 21:49:48.826267 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:48.826089    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f339f5ae44f471c7abb2599296584ab7a73c807bafa55dbade527725ac6c4c4"
Oct 30 21:49:49.551902 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.551875    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:49:49.628043 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.628018    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00878b67-261a-4b77-ac91-a29de49126ae-kube-api-access-gvr94\" (UniqueName: \"kubernetes.io/projected/00878b67-261a-4b77-ac91-a29de49126ae-kube-api-access-gvr94\") pod \"00878b67-261a-4b77-ac91-a29de49126ae\" (UID: \"00878b67-261a-4b77-ac91-a29de49126ae\") "
Oct 30 21:49:49.628197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.628182    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00878b67-261a-4b77-ac91-a29de49126ae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00878b67-261a-4b77-ac91-a29de49126ae-plugins-config-volume\") pod \"00878b67-261a-4b77-ac91-a29de49126ae\" (UID: \"00878b67-261a-4b77-ac91-a29de49126ae\") "
Oct 30 21:49:49.628526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.628504    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00878b67-261a-4b77-ac91-a29de49126ae-plugins-config-volume" pod "00878b67-261a-4b77-ac91-a29de49126ae" (UID: "00878b67-261a-4b77-ac91-a29de49126ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:49.628622 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.628612    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00878b67-261a-4b77-ac91-a29de49126ae-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:49.630249 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.630200    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00878b67-261a-4b77-ac91-a29de49126ae-kube-api-access-gvr94" pod "00878b67-261a-4b77-ac91-a29de49126ae" (UID: "00878b67-261a-4b77-ac91-a29de49126ae"). InnerVolumeSpecName "kube-api-access-gvr94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:49.631372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.631355    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:49:49.729261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.729235    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/903ce8c0-dd05-491c-a34f-344fd7275b31-kube-api-access-4pjbh\" (UniqueName: \"kubernetes.io/projected/903ce8c0-dd05-491c-a34f-344fd7275b31-kube-api-access-4pjbh\") pod \"903ce8c0-dd05-491c-a34f-344fd7275b31\" (UID: \"903ce8c0-dd05-491c-a34f-344fd7275b31\") "
Oct 30 21:49:49.729384 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.729310    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/903ce8c0-dd05-491c-a34f-344fd7275b31-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/903ce8c0-dd05-491c-a34f-344fd7275b31-plugins-config-volume\") pod \"903ce8c0-dd05-491c-a34f-344fd7275b31\" (UID: \"903ce8c0-dd05-491c-a34f-344fd7275b31\") "
Oct 30 21:49:49.729671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.729652    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/903ce8c0-dd05-491c-a34f-344fd7275b31-plugins-config-volume" pod "903ce8c0-dd05-491c-a34f-344fd7275b31" (UID: "903ce8c0-dd05-491c-a34f-344fd7275b31"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:49.729743 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.729663    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvr94\" (UniqueName: \"kubernetes.io/projected/00878b67-261a-4b77-ac91-a29de49126ae-kube-api-access-gvr94\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:49.731340 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.731318    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/903ce8c0-dd05-491c-a34f-344fd7275b31-kube-api-access-4pjbh" pod "903ce8c0-dd05-491c-a34f-344fd7275b31" (UID: "903ce8c0-dd05-491c-a34f-344fd7275b31"). InnerVolumeSpecName "kube-api-access-4pjbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:49.830535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.830463    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/903ce8c0-dd05-491c-a34f-344fd7275b31-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:49.830535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.830497    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pjbh\" (UniqueName: \"kubernetes.io/projected/903ce8c0-dd05-491c-a34f-344fd7275b31-kube-api-access-4pjbh\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:49.834466 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.834441    5786 generic.go:356] "Generic (PLEG): container finished" podID="903ce8c0-dd05-491c-a34f-344fd7275b31" containerID="c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234" exitCode=0
Oct 30 21:49:49.834572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.834488    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"
Oct 30 21:49:49.834572 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.834529    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b" event={"ID":"903ce8c0-dd05-491c-a34f-344fd7275b31","Type":"ContainerDied","Data":"c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234"}
Oct 30 21:49:49.834648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.834575    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b" event={"ID":"903ce8c0-dd05-491c-a34f-344fd7275b31","Type":"ContainerDied","Data":"8e0cc54c8f7be396d87e4754acceee502098a20e4c220f792ae937b3e255fb64"}
Oct 30 21:49:49.834648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.834599    5786 scope.go:122] "RemoveContainer" containerID="c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234"
Oct 30 21:49:49.836057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.836031    5786 generic.go:356] "Generic (PLEG): container finished" podID="00878b67-261a-4b77-ac91-a29de49126ae" containerID="d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6" exitCode=0
Oct 30 21:49:49.836164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.836092    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"
Oct 30 21:49:49.836164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.836142    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58" event={"ID":"00878b67-261a-4b77-ac91-a29de49126ae","Type":"ContainerDied","Data":"d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6"}
Oct 30 21:49:49.836302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.836185    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58" event={"ID":"00878b67-261a-4b77-ac91-a29de49126ae","Type":"ContainerDied","Data":"076ec678c578cc42695c13f8b51447fbbbf9302915ef32f5440641fe0b4b93e1"}
Oct 30 21:49:49.844331 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.844316    5786 scope.go:122] "RemoveContainer" containerID="c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234"
Oct 30 21:49:49.844649 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:49.844617    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234\": container with ID starting with c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234 not found: ID does not exist" containerID="c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234"
Oct 30 21:49:49.844746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.844658    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234"} err="failed to get container status \"c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234\": rpc error: code = NotFound desc = could not find container \"c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234\": container with ID starting with c1a28d884a2dd6d2c712c7ecaad9d33c661663a076662de086fbf8accd0ea234 not found: ID does not exist"
Oct 30 21:49:49.844746 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.844681    5786 scope.go:122] "RemoveContainer" containerID="d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6"
Oct 30 21:49:49.853908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.853891    5786 scope.go:122] "RemoveContainer" containerID="d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6"
Oct 30 21:49:49.854181 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:49.854157    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6\": container with ID starting with d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6 not found: ID does not exist" containerID="d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6"
Oct 30 21:49:49.854285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.854186    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6"} err="failed to get container status \"d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6\": rpc error: code = NotFound desc = could not find container \"d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6\": container with ID starting with d9dc2205eb19edef296474f0d8da9bc49004f4e095d8a62136dc74cd32eea5c6 not found: ID does not exist"
Oct 30 21:49:49.996971 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:49.996945    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"]
Oct 30 21:49:50.002908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.002887    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3ab88248-58b75f5f87-pbt9b"]
Oct 30 21:49:50.109304 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.109239    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"]
Oct 30 21:49:50.112667 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.112644    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ab88248-7f6d44cc95-68x58"]
Oct 30 21:49:50.359037 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.359010    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00878b67-261a-4b77-ac91-a29de49126ae" path="/var/lib/kubelet/pods/00878b67-261a-4b77-ac91-a29de49126ae/volumes"
Oct 30 21:49:50.359448 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.359410    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="903ce8c0-dd05-491c-a34f-344fd7275b31" path="/var/lib/kubelet/pods/903ce8c0-dd05-491c-a34f-344fd7275b31/volumes"
Oct 30 21:49:50.545255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.545218    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk" podUID="80159d30-c714-496e-b901-38bb4ce46d8d" containerName="payload-processing" containerID="cri-o://e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee" gracePeriod=25
Oct 30 21:49:50.653700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.653646    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m" podUID="9c3fc218-76e9-41e6-80be-1b44f16d4110" containerName="payload-pre-processing" containerID="cri-o://4e57f21793ea8b70518962f6e8cc62039c5a9c9a30f4ebc7d3ea4f967cb36ff9" gracePeriod=25
Oct 30 21:49:50.817414 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.817392    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:49:50.845426 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.845394    5786 generic.go:356] "Generic (PLEG): container finished" podID="80159d30-c714-496e-b901-38bb4ce46d8d" containerID="e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee" exitCode=0
Oct 30 21:49:50.845557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.845444    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"
Oct 30 21:49:50.845557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.845462    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk" event={"ID":"80159d30-c714-496e-b901-38bb4ce46d8d","Type":"ContainerDied","Data":"e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee"}
Oct 30 21:49:50.845557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.845510    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk" event={"ID":"80159d30-c714-496e-b901-38bb4ce46d8d","Type":"ContainerDied","Data":"ae81c43c367add235100651094f2cd215b06937cf33ba90802c77beb81fe82ba"}
Oct 30 21:49:50.845557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.845524    5786 scope.go:122] "RemoveContainer" containerID="e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee"
Oct 30 21:49:50.852475 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.852451    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80159d30-c714-496e-b901-38bb4ce46d8d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159d30-c714-496e-b901-38bb4ce46d8d-plugins-config-volume\") pod \"80159d30-c714-496e-b901-38bb4ce46d8d\" (UID: \"80159d30-c714-496e-b901-38bb4ce46d8d\") "
Oct 30 21:49:50.852754 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.852737    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80159d30-c714-496e-b901-38bb4ce46d8d-kube-api-access-rr59q\" (UniqueName: \"kubernetes.io/projected/80159d30-c714-496e-b901-38bb4ce46d8d-kube-api-access-rr59q\") pod \"80159d30-c714-496e-b901-38bb4ce46d8d\" (UID: \"80159d30-c714-496e-b901-38bb4ce46d8d\") "
Oct 30 21:49:50.852878 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.852850    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80159d30-c714-496e-b901-38bb4ce46d8d-plugins-config-volume" pod "80159d30-c714-496e-b901-38bb4ce46d8d" (UID: "80159d30-c714-496e-b901-38bb4ce46d8d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:50.853005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.852980    5786 generic.go:356] "Generic (PLEG): container finished" podID="9c3fc218-76e9-41e6-80be-1b44f16d4110" containerID="4e57f21793ea8b70518962f6e8cc62039c5a9c9a30f4ebc7d3ea4f967cb36ff9" exitCode=0
Oct 30 21:49:50.853141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.853025    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m" event={"ID":"9c3fc218-76e9-41e6-80be-1b44f16d4110","Type":"ContainerDied","Data":"4e57f21793ea8b70518962f6e8cc62039c5a9c9a30f4ebc7d3ea4f967cb36ff9"}
Oct 30 21:49:50.853507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.853484    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159d30-c714-496e-b901-38bb4ce46d8d-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:50.855638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.855617    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80159d30-c714-496e-b901-38bb4ce46d8d-kube-api-access-rr59q" pod "80159d30-c714-496e-b901-38bb4ce46d8d" (UID: "80159d30-c714-496e-b901-38bb4ce46d8d"). InnerVolumeSpecName "kube-api-access-rr59q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:50.856620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.856604    5786 scope.go:122] "RemoveContainer" containerID="e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee"
Oct 30 21:49:50.856881 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:50.856864    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee\": container with ID starting with e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee not found: ID does not exist" containerID="e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee"
Oct 30 21:49:50.856967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.856891    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee"} err="failed to get container status \"e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee\": rpc error: code = NotFound desc = could not find container \"e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee\": container with ID starting with e4a58bc97e86b48ea93de5b3968459d01bfce8aaa0db836be4f89cca3de2f6ee not found: ID does not exist"
Oct 30 21:49:50.920750 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.920732    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:49:50.954505 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.954482    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c3fc218-76e9-41e6-80be-1b44f16d4110-kube-api-access-m7swf\" (UniqueName: \"kubernetes.io/projected/9c3fc218-76e9-41e6-80be-1b44f16d4110-kube-api-access-m7swf\") pod \"9c3fc218-76e9-41e6-80be-1b44f16d4110\" (UID: \"9c3fc218-76e9-41e6-80be-1b44f16d4110\") "
Oct 30 21:49:50.954641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.954524    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c3fc218-76e9-41e6-80be-1b44f16d4110-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3fc218-76e9-41e6-80be-1b44f16d4110-plugins-config-volume\") pod \"9c3fc218-76e9-41e6-80be-1b44f16d4110\" (UID: \"9c3fc218-76e9-41e6-80be-1b44f16d4110\") "
Oct 30 21:49:50.954970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.954940    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c3fc218-76e9-41e6-80be-1b44f16d4110-plugins-config-volume" pod "9c3fc218-76e9-41e6-80be-1b44f16d4110" (UID: "9c3fc218-76e9-41e6-80be-1b44f16d4110"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:50.955124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.955099    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3fc218-76e9-41e6-80be-1b44f16d4110-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:50.955124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.955119    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr59q\" (UniqueName: \"kubernetes.io/projected/80159d30-c714-496e-b901-38bb4ce46d8d-kube-api-access-rr59q\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:50.956688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:50.956666    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3fc218-76e9-41e6-80be-1b44f16d4110-kube-api-access-m7swf" pod "9c3fc218-76e9-41e6-80be-1b44f16d4110" (UID: "9c3fc218-76e9-41e6-80be-1b44f16d4110"). InnerVolumeSpecName "kube-api-access-m7swf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:51.056370 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:51.056348    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7swf\" (UniqueName: \"kubernetes.io/projected/9c3fc218-76e9-41e6-80be-1b44f16d4110-kube-api-access-m7swf\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:51.264737 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:51.264712    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"]
Oct 30 21:49:51.268502 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:51.268480    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9085f4-79dd574f5f-lklmk"]
Oct 30 21:49:51.861881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:51.861850    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m" event={"ID":"9c3fc218-76e9-41e6-80be-1b44f16d4110","Type":"ContainerDied","Data":"fceb091a6cb16803b094e44c8c2eac35a76e0604ff712e49174b75a2f57481df"}
Oct 30 21:49:51.861881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:51.861879    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"
Oct 30 21:49:51.862396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:51.861907    5786 scope.go:122] "RemoveContainer" containerID="4e57f21793ea8b70518962f6e8cc62039c5a9c9a30f4ebc7d3ea4f967cb36ff9"
Oct 30 21:49:52.023124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:52.023100    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"]
Oct 30 21:49:52.028393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:52.028373    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9085f4-6b7999f64c-f9z9m"]
Oct 30 21:49:52.353927 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:49:52.353893    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:49:52.358869 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:52.358844    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80159d30-c714-496e-b901-38bb4ce46d8d" path="/var/lib/kubelet/pods/80159d30-c714-496e-b901-38bb4ce46d8d/volumes"
Oct 30 21:49:52.359290 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:52.359276    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c3fc218-76e9-41e6-80be-1b44f16d4110" path="/var/lib/kubelet/pods/9c3fc218-76e9-41e6-80be-1b44f16d4110/volumes"
Oct 30 21:49:53.048700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:53.048667    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"]
Oct 30 21:49:53.097533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:53.097505    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3ed0a6b-pqxls"]
Oct 30 21:49:54.049057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:54.048985    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"]
Oct 30 21:49:54.097892 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:54.097855    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-14c2cd68-vv5qx"]
Oct 30 21:49:54.358381 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:54.358306    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e333545-99c8-4bc6-a8e8-4be47f9ed805" path="/var/lib/kubelet/pods/0e333545-99c8-4bc6-a8e8-4be47f9ed805/volumes"
Oct 30 21:49:54.358790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:49:54.358768    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b814d735-eaf1-4031-b2de-eaad057d8d8d" path="/var/lib/kubelet/pods/b814d735-eaf1-4031-b2de-eaad057d8d8d/volumes"
Oct 30 21:50:00.895251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:00.895152    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"]
Oct 30 21:50:00.943828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:00.943794    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-9085f4-9db29"]
Oct 30 21:50:02.359887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:02.359858    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73990eb0-b383-4131-85e5-5dd7b6bdf6a5" path="/var/lib/kubelet/pods/73990eb0-b383-4131-85e5-5dd7b6bdf6a5/volumes"
Oct 30 21:50:09.414660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.414626    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"]
Oct 30 21:50:09.415247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415229    5786 cpu_manager.go:414] "containerMap: removing container" podUID="9c3fc218-76e9-41e6-80be-1b44f16d4110" containerName="payload-pre-processing"
Oct 30 21:50:09.415315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415247    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c3fc218-76e9-41e6-80be-1b44f16d4110" containerName="payload-pre-processing"
Oct 30 21:50:09.415315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415259    5786 cpu_manager.go:414] "containerMap: removing container" podUID="568d4be3-420b-4b37-af8b-51474561b8dc" containerName="revoke-keys"
Oct 30 21:50:09.415315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415265    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="568d4be3-420b-4b37-af8b-51474561b8dc" containerName="revoke-keys"
Oct 30 21:50:09.415315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415296    5786 cpu_manager.go:414] "containerMap: removing container" podUID="80159d30-c714-496e-b901-38bb4ce46d8d" containerName="payload-processing"
Oct 30 21:50:09.415315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415301    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80159d30-c714-496e-b901-38bb4ce46d8d" containerName="payload-processing"
Oct 30 21:50:09.415315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415309    5786 cpu_manager.go:414] "containerMap: removing container" podUID="00878b67-261a-4b77-ac91-a29de49126ae" containerName="payload-processing"
Oct 30 21:50:09.415315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415313    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00878b67-261a-4b77-ac91-a29de49126ae" containerName="payload-processing"
Oct 30 21:50:09.415573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415321    5786 cpu_manager.go:414] "containerMap: removing container" podUID="903ce8c0-dd05-491c-a34f-344fd7275b31" containerName="payload-pre-processing"
Oct 30 21:50:09.415573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415325    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="903ce8c0-dd05-491c-a34f-344fd7275b31" containerName="payload-pre-processing"
Oct 30 21:50:09.415573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415398    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="80159d30-c714-496e-b901-38bb4ce46d8d" containerName="payload-processing"
Oct 30 21:50:09.415573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415407    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="00878b67-261a-4b77-ac91-a29de49126ae" containerName="payload-processing"
Oct 30 21:50:09.415573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415414    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="903ce8c0-dd05-491c-a34f-344fd7275b31" containerName="payload-pre-processing"
Oct 30 21:50:09.415573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415423    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c3fc218-76e9-41e6-80be-1b44f16d4110" containerName="payload-pre-processing"
Oct 30 21:50:09.415573 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.415430    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="568d4be3-420b-4b37-af8b-51474561b8dc" containerName="revoke-keys"
Oct 30 21:50:09.420158 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.420143    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.476536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.476509    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"]
Oct 30 21:50:09.522955 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.522928    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:50:09.523078 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.522952    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:50:09.550775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.550753    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.550915 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.550865    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.550991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.550975    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klkzj\" (UniqueName: \"kubernetes.io/projected/5dc1ec42-7e52-481e-b78a-25504649ce10-kube-api-access-klkzj\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.652147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.652112    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klkzj\" (UniqueName: \"kubernetes.io/projected/5dc1ec42-7e52-481e-b78a-25504649ce10-kube-api-access-klkzj\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.652343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.652239    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.652343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.652275    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.652850 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.652819    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.654635 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.654614    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.707664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.707614    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klkzj\" (UniqueName: \"kubernetes.io/projected/5dc1ec42-7e52-481e-b78a-25504649ce10-kube-api-access-klkzj\") pod \"maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:09.733064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:09.733043    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:10.108392 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:10.108368    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"]
Oct 30 21:50:10.110643 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:50:10.110620    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc1ec42_7e52_481e_b78a_25504649ce10.slice/crio-3b2cfc28b11b465ee6afac88f2951c0e3561f274f9ed7972a8d35f19e8d619bd WatchSource:0}: Error finding container 3b2cfc28b11b465ee6afac88f2951c0e3561f274f9ed7972a8d35f19e8d619bd: Status 404 returned error can't find the container with id 3b2cfc28b11b465ee6afac88f2951c0e3561f274f9ed7972a8d35f19e8d619bd
Oct 30 21:50:10.980740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:10.980716    5786 generic.go:356] "Generic (PLEG): container finished" podID="5dc1ec42-7e52-481e-b78a-25504649ce10" containerID="21815cd86d3ddee164702e5c699861d78a2777c1debd9c46fb2135ae7daf44d3" exitCode=0
Oct 30 21:50:10.981124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:10.980807    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz" event={"ID":"5dc1ec42-7e52-481e-b78a-25504649ce10","Type":"ContainerDied","Data":"21815cd86d3ddee164702e5c699861d78a2777c1debd9c46fb2135ae7daf44d3"}
Oct 30 21:50:10.981124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:10.980856    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz" event={"ID":"5dc1ec42-7e52-481e-b78a-25504649ce10","Type":"ContainerStarted","Data":"3b2cfc28b11b465ee6afac88f2951c0e3561f274f9ed7972a8d35f19e8d619bd"}
Oct 30 21:50:12.116343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.116320    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:12.177549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.177524    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dc1ec42-7e52-481e-b78a-25504649ce10-kube-api-access-klkzj\" (UniqueName: \"kubernetes.io/projected/5dc1ec42-7e52-481e-b78a-25504649ce10-kube-api-access-klkzj\") pod \"5dc1ec42-7e52-481e-b78a-25504649ce10\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") "
Oct 30 21:50:12.177663 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.177627    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-openshift-ca\") pod \"5dc1ec42-7e52-481e-b78a-25504649ce10\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") "
Oct 30 21:50:12.177722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.177711    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-xks-ca\") pod \"5dc1ec42-7e52-481e-b78a-25504649ce10\" (UID: \"5dc1ec42-7e52-481e-b78a-25504649ce10\") "
Oct 30 21:50:12.177978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.177951    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-openshift-ca" pod "5dc1ec42-7e52-481e-b78a-25504649ce10" (UID: "5dc1ec42-7e52-481e-b78a-25504649ce10"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:12.178167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.178151    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:12.179508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.179489    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc1ec42-7e52-481e-b78a-25504649ce10-kube-api-access-klkzj" pod "5dc1ec42-7e52-481e-b78a-25504649ce10" (UID: "5dc1ec42-7e52-481e-b78a-25504649ce10"). InnerVolumeSpecName "kube-api-access-klkzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:12.179824 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.179807    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-xks-ca" pod "5dc1ec42-7e52-481e-b78a-25504649ce10" (UID: "5dc1ec42-7e52-481e-b78a-25504649ce10"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:12.279486 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.279426    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dc1ec42-7e52-481e-b78a-25504649ce10-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:12.279486 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.279448    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klkzj\" (UniqueName: \"kubernetes.io/projected/5dc1ec42-7e52-481e-b78a-25504649ce10-kube-api-access-klkzj\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:12.992968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.992945    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"
Oct 30 21:50:12.993140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.992964    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz" event={"ID":"5dc1ec42-7e52-481e-b78a-25504649ce10","Type":"ContainerDied","Data":"3b2cfc28b11b465ee6afac88f2951c0e3561f274f9ed7972a8d35f19e8d619bd"}
Oct 30 21:50:12.993140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:12.993011    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b2cfc28b11b465ee6afac88f2951c0e3561f274f9ed7972a8d35f19e8d619bd"
Oct 30 21:50:13.936598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:13.936569    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"]
Oct 30 21:50:17.117271 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:17.117242    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"]
Oct 30 21:50:17.165967 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:17.165944    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-tqcf4"]
Oct 30 21:50:18.359251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:18.359194    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a" path="/var/lib/kubelet/pods/bcfc1aa2-ba05-4f20-a37c-bbd2b466ff5a/volumes"
Oct 30 21:50:20.904565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:20.904529    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"]
Oct 30 21:50:21.024767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:21.024734    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"]
Oct 30 21:50:22.399510 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:50:22.399443    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:50:25.905095 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:25.905045    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4" podUID="b9679d46-0e10-4523-af31-7362a4871af7" containerName="payload-processing" containerID="cri-o://23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd" gracePeriod=25
Oct 30 21:50:26.025535 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.025484    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5" podUID="16e6d78c-7d7b-4c62-be32-5808590e1345" containerName="payload-pre-processing" containerID="cri-o://1b0178b3915c4284ef636216e222be7830d4a9d16854bbbb632e508f7724e62d" gracePeriod=25
Oct 30 21:50:26.391660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.391624    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"]
Oct 30 21:50:26.392234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.392189    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5dc1ec42-7e52-481e-b78a-25504649ce10" containerName="revoke-keys"
Oct 30 21:50:26.392234 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.392217    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dc1ec42-7e52-481e-b78a-25504649ce10" containerName="revoke-keys"
Oct 30 21:50:26.392343 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.392301    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dc1ec42-7e52-481e-b78a-25504649ce10" containerName="revoke-keys"
Oct 30 21:50:26.395778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.395758    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.451916 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.451895    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"]
Oct 30 21:50:26.498010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.497988    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:50:26.498010 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.497990    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:50:26.534858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.534834    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.534959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.534868    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79tgk\" (UniqueName: \"kubernetes.io/projected/7bc433cb-99ff-4a5e-a250-00dfa44ed156-kube-api-access-79tgk\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.535021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.535000    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.636630 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.636602    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.636758 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.636642    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79tgk\" (UniqueName: \"kubernetes.io/projected/7bc433cb-99ff-4a5e-a250-00dfa44ed156-kube-api-access-79tgk\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.636758 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.636702    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.637315 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.637297    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.639091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.639069    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.693709 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.693691    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79tgk\" (UniqueName: \"kubernetes.io/projected/7bc433cb-99ff-4a5e-a250-00dfa44ed156-kube-api-access-79tgk\") pod \"maas-api-revoke-sub-a01485f4-smdff\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:26.708375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:26.708352    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:27.139261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:27.132872    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"]
Oct 30 21:50:27.151125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:27.151090    5786 scope.go:122] "RemoveContainer" containerID="e7d847181151b3e1fde439214e9139970580de3473705213e4861b64a85dc0dd"
Oct 30 21:50:27.153575 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:50:27.153540    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc433cb_99ff_4a5e_a250_00dfa44ed156.slice/crio-955c21c43ace7a863177c12c01bf7410df6720658db4dd4619a638cd16f3bdc7 WatchSource:0}: Error finding container 955c21c43ace7a863177c12c01bf7410df6720658db4dd4619a638cd16f3bdc7: Status 404 returned error can't find the container with id 955c21c43ace7a863177c12c01bf7410df6720658db4dd4619a638cd16f3bdc7
Oct 30 21:50:28.095795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:28.095768    5786 generic.go:356] "Generic (PLEG): container finished" podID="7bc433cb-99ff-4a5e-a250-00dfa44ed156" containerID="b3a25dbd8aebbda97fddb50bd5a4a49d7dbe055dec98d794ec4048edcf125c5c" exitCode=0
Oct 30 21:50:28.095959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:28.095849    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff" event={"ID":"7bc433cb-99ff-4a5e-a250-00dfa44ed156","Type":"ContainerDied","Data":"b3a25dbd8aebbda97fddb50bd5a4a49d7dbe055dec98d794ec4048edcf125c5c"}
Oct 30 21:50:28.095959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:28.095915    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff" event={"ID":"7bc433cb-99ff-4a5e-a250-00dfa44ed156","Type":"ContainerStarted","Data":"955c21c43ace7a863177c12c01bf7410df6720658db4dd4619a638cd16f3bdc7"}
Oct 30 21:50:28.956886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:28.956829    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" containerID="cri-o://5722cfc26c816eb40f4ce7a2296e16a99a599311315151ec3c466d40bac78d17" gracePeriod=45
Oct 30 21:50:28.961526 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:28.961498    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": EOF"
Oct 30 21:50:29.104838 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.104812    5786 generic.go:356] "Generic (PLEG): container finished" podID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerID="5722cfc26c816eb40f4ce7a2296e16a99a599311315151ec3c466d40bac78d17" exitCode=0
Oct 30 21:50:29.105009 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.104889    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" event={"ID":"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8","Type":"ContainerDied","Data":"5722cfc26c816eb40f4ce7a2296e16a99a599311315151ec3c466d40bac78d17"}
Oct 30 21:50:29.275333 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.275305    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:50:29.296263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.296242    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:29.366789 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.366763    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-model-cache\") pod \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") "
Oct 30 21:50:29.366928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.366850    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tmp-dir\") pod \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") "
Oct 30 21:50:29.366928 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.366900    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-dshm\") pod \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") "
Oct 30 21:50:29.367050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.366953    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-kube-api-access-8n6c4\" (UniqueName: \"kubernetes.io/projected/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-kube-api-access-8n6c4\") pod \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") "
Oct 30 21:50:29.367050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.366976    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-xks-ca\") pod \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") "
Oct 30 21:50:29.367050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.366977    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-model-cache" pod "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" (UID: "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:29.367050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.366993    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tmp-dir" pod "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" (UID: "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:29.367050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367010    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bc433cb-99ff-4a5e-a250-00dfa44ed156-kube-api-access-79tgk\" (UniqueName: \"kubernetes.io/projected/7bc433cb-99ff-4a5e-a250-00dfa44ed156-kube-api-access-79tgk\") pod \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") "
Oct 30 21:50:29.367366 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367059    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-openshift-ca\") pod \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\" (UID: \"7bc433cb-99ff-4a5e-a250-00dfa44ed156\") "
Oct 30 21:50:29.367366 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367082    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-home\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-home\") pod \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") "
Oct 30 21:50:29.367366 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367170    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tls-certs\" (UniqueName: \"kubernetes.io/secret/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tls-certs\") pod \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\" (UID: \"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8\") "
Oct 30 21:50:29.367525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367478    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-openshift-ca" pod "7bc433cb-99ff-4a5e-a250-00dfa44ed156" (UID: "7bc433cb-99ff-4a5e-a250-00dfa44ed156"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:29.367642 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367594    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:29.367827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367619    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:29.367827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367698    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:29.367940 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.367854    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-home" pod "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" (UID: "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:29.369286 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.369266    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bc433cb-99ff-4a5e-a250-00dfa44ed156-kube-api-access-79tgk" pod "7bc433cb-99ff-4a5e-a250-00dfa44ed156" (UID: "7bc433cb-99ff-4a5e-a250-00dfa44ed156"). InnerVolumeSpecName "kube-api-access-79tgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:29.369401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.369374    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tls-certs" pod "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" (UID: "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:29.369401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.369377    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-dshm" pod "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" (UID: "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:29.369731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.369709    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-xks-ca" pod "7bc433cb-99ff-4a5e-a250-00dfa44ed156" (UID: "7bc433cb-99ff-4a5e-a250-00dfa44ed156"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:29.369825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.369729    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-kube-api-access-8n6c4" pod "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" (UID: "61f2966e-e0d9-44b0-8ac5-69d7a678a0d8"). InnerVolumeSpecName "kube-api-access-8n6c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:29.469675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.469555    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:29.469675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.469585    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:29.469675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.469607    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n6c4\" (UniqueName: \"kubernetes.io/projected/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-kube-api-access-8n6c4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:29.469675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.469623    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7bc433cb-99ff-4a5e-a250-00dfa44ed156-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:29.469675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.469636    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79tgk\" (UniqueName: \"kubernetes.io/projected/7bc433cb-99ff-4a5e-a250-00dfa44ed156-kube-api-access-79tgk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:29.469675 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:29.469649    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:30.111899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:30.111875    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"
Oct 30 21:50:30.112375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:30.111913    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff" event={"ID":"7bc433cb-99ff-4a5e-a250-00dfa44ed156","Type":"ContainerDied","Data":"955c21c43ace7a863177c12c01bf7410df6720658db4dd4619a638cd16f3bdc7"}
Oct 30 21:50:30.112375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:30.111957    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="955c21c43ace7a863177c12c01bf7410df6720658db4dd4619a638cd16f3bdc7"
Oct 30 21:50:30.113308 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:30.113287    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"
Oct 30 21:50:30.113455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:30.113325    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr" event={"ID":"61f2966e-e0d9-44b0-8ac5-69d7a678a0d8","Type":"ContainerDied","Data":"28da04c2497cb189d2e816d816b995285cbeefc4edf42a20a8033b972a3ed8c5"}
Oct 30 21:50:30.113455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:30.113366    5786 scope.go:122] "RemoveContainer" containerID="5722cfc26c816eb40f4ce7a2296e16a99a599311315151ec3c466d40bac78d17"
Oct 30 21:50:30.325684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:30.325625    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"]
Oct 30 21:50:30.374710 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:30.374685    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-be9451/premium-w0-be9451-kserve-79645b5648-8hwxr"]
Oct 30 21:50:32.051000 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:32.050963    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"]
Oct 30 21:50:32.088443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:32.088412    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66b312e3-zmcbp"]
Oct 30 21:50:32.359116 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:32.359025    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38f2fbc5-0038-4309-b84b-890c561871cf" path="/var/lib/kubelet/pods/38f2fbc5-0038-4309-b84b-890c561871cf/volumes"
Oct 30 21:50:32.359594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:32.359573    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" path="/var/lib/kubelet/pods/61f2966e-e0d9-44b0-8ac5-69d7a678a0d8/volumes"
Oct 30 21:50:33.654951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:33.654918    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4" podUID="b9679d46-0e10-4523-af31-7362a4871af7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:50:33.655355 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:33.654957    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5" podUID="16e6d78c-7d7b-4c62-be32-5808590e1345" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:50:43.654899 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:43.654866    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4" podUID="b9679d46-0e10-4523-af31-7362a4871af7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:50:43.655413 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:43.654878    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5" podUID="16e6d78c-7d7b-4c62-be32-5808590e1345" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:50:46.181312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.181286    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:50:46.216559 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.216526    5786 generic.go:356] "Generic (PLEG): container finished" podID="b9679d46-0e10-4523-af31-7362a4871af7" containerID="23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd" exitCode=0
Oct 30 21:50:46.216790 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.216776    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"
Oct 30 21:50:46.216914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.216899    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4" event={"ID":"b9679d46-0e10-4523-af31-7362a4871af7","Type":"ContainerDied","Data":"23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd"}
Oct 30 21:50:46.216984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.216933    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4" event={"ID":"b9679d46-0e10-4523-af31-7362a4871af7","Type":"ContainerDied","Data":"e0d77e2789981ecd8b92d40760320618e67880db7093479cccfd5c45da2f2f29"}
Oct 30 21:50:46.216984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.216954    5786 scope.go:122] "RemoveContainer" containerID="23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd"
Oct 30 21:50:46.218898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.218878    5786 generic.go:356] "Generic (PLEG): container finished" podID="16e6d78c-7d7b-4c62-be32-5808590e1345" containerID="1b0178b3915c4284ef636216e222be7830d4a9d16854bbbb632e508f7724e62d" exitCode=0
Oct 30 21:50:46.219023 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.218915    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5" event={"ID":"16e6d78c-7d7b-4c62-be32-5808590e1345","Type":"ContainerDied","Data":"1b0178b3915c4284ef636216e222be7830d4a9d16854bbbb632e508f7724e62d"}
Oct 30 21:50:46.229352 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.229304    5786 scope.go:122] "RemoveContainer" containerID="23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd"
Oct 30 21:50:46.229668 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:50:46.229648    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd\": container with ID starting with 23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd not found: ID does not exist" containerID="23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd"
Oct 30 21:50:46.229734 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.229679    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd"} err="failed to get container status \"23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd\": rpc error: code = NotFound desc = could not find container \"23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd\": container with ID starting with 23ead5e8ebe0948643d4ff01ad19a1bbe946b64f9db5dfff38beb1f8a917e3dd not found: ID does not exist"
Oct 30 21:50:46.242582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.242557    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9679d46-0e10-4523-af31-7362a4871af7-kube-api-access-x7bzl\" (UniqueName: \"kubernetes.io/projected/b9679d46-0e10-4523-af31-7362a4871af7-kube-api-access-x7bzl\") pod \"b9679d46-0e10-4523-af31-7362a4871af7\" (UID: \"b9679d46-0e10-4523-af31-7362a4871af7\") "
Oct 30 21:50:46.242700 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.242627    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9679d46-0e10-4523-af31-7362a4871af7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9679d46-0e10-4523-af31-7362a4871af7-plugins-config-volume\") pod \"b9679d46-0e10-4523-af31-7362a4871af7\" (UID: \"b9679d46-0e10-4523-af31-7362a4871af7\") "
Oct 30 21:50:46.242997 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.242973    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9679d46-0e10-4523-af31-7362a4871af7-plugins-config-volume" pod "b9679d46-0e10-4523-af31-7362a4871af7" (UID: "b9679d46-0e10-4523-af31-7362a4871af7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:46.243146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.243130    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9679d46-0e10-4523-af31-7362a4871af7-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.248263 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.246334    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9679d46-0e10-4523-af31-7362a4871af7-kube-api-access-x7bzl" pod "b9679d46-0e10-4523-af31-7362a4871af7" (UID: "b9679d46-0e10-4523-af31-7362a4871af7"). InnerVolumeSpecName "kube-api-access-x7bzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:46.272532 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.272512    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:50:46.344123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.344101    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16e6d78c-7d7b-4c62-be32-5808590e1345-kube-api-access-pq4vp\" (UniqueName: \"kubernetes.io/projected/16e6d78c-7d7b-4c62-be32-5808590e1345-kube-api-access-pq4vp\") pod \"16e6d78c-7d7b-4c62-be32-5808590e1345\" (UID: \"16e6d78c-7d7b-4c62-be32-5808590e1345\") "
Oct 30 21:50:46.344279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.344143    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16e6d78c-7d7b-4c62-be32-5808590e1345-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16e6d78c-7d7b-4c62-be32-5808590e1345-plugins-config-volume\") pod \"16e6d78c-7d7b-4c62-be32-5808590e1345\" (UID: \"16e6d78c-7d7b-4c62-be32-5808590e1345\") "
Oct 30 21:50:46.344437 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.344423    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7bzl\" (UniqueName: \"kubernetes.io/projected/b9679d46-0e10-4523-af31-7362a4871af7-kube-api-access-x7bzl\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.344566 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.344541    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16e6d78c-7d7b-4c62-be32-5808590e1345-plugins-config-volume" pod "16e6d78c-7d7b-4c62-be32-5808590e1345" (UID: "16e6d78c-7d7b-4c62-be32-5808590e1345"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:46.346283 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.346264    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16e6d78c-7d7b-4c62-be32-5808590e1345-kube-api-access-pq4vp" pod "16e6d78c-7d7b-4c62-be32-5808590e1345" (UID: "16e6d78c-7d7b-4c62-be32-5808590e1345"). InnerVolumeSpecName "kube-api-access-pq4vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:46.446770 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.446585    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq4vp\" (UniqueName: \"kubernetes.io/projected/16e6d78c-7d7b-4c62-be32-5808590e1345-kube-api-access-pq4vp\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.446770 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.446624    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16e6d78c-7d7b-4c62-be32-5808590e1345-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.629309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.629278    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"]
Oct 30 21:50:46.631387 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:46.631368    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-be9451-5f6d8cc7cf-wbtg4"]
Oct 30 21:50:47.226569 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:47.226500    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"
Oct 30 21:50:47.226985 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:47.226510    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5" event={"ID":"16e6d78c-7d7b-4c62-be32-5808590e1345","Type":"ContainerDied","Data":"0b4248b77ed33550b21477d0e458626a729f7701b5b2f57a7772c5557a686f75"}
Oct 30 21:50:47.226985 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:47.226628    5786 scope.go:122] "RemoveContainer" containerID="1b0178b3915c4284ef636216e222be7830d4a9d16854bbbb632e508f7724e62d"
Oct 30 21:50:47.391390 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:47.391366    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"]
Oct 30 21:50:47.440840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:47.440813    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-be9451-7786c79c59-tq8k5"]
Oct 30 21:50:48.358482 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:48.358456    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16e6d78c-7d7b-4c62-be32-5808590e1345" path="/var/lib/kubelet/pods/16e6d78c-7d7b-4c62-be32-5808590e1345/volumes"
Oct 30 21:50:48.358894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:48.358806    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9679d46-0e10-4523-af31-7362a4871af7" path="/var/lib/kubelet/pods/b9679d46-0e10-4523-af31-7362a4871af7/volumes"
Oct 30 21:50:52.446824 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:50:52.446789    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:50:52.588285 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:52.588251    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"]
Oct 30 21:50:52.591984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:52.591957    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-be9451-6x5qz"]
Oct 30 21:50:54.358575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:50:54.358544    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5dc1ec42-7e52-481e-b78a-25504649ce10" path="/var/lib/kubelet/pods/5dc1ec42-7e52-481e-b78a-25504649ce10/volumes"
Oct 30 21:51:00.313365 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:00.313337    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"]
Oct 30 21:51:00.313886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:00.313538    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" containerID="cri-o://7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201" gracePeriod=30
Oct 30 21:51:00.325802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:00.325775    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"]
Oct 30 21:51:00.326001 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:00.325963    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy" containerID="cri-o://4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e" gracePeriod=30
Oct 30 21:51:01.886470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:01.886424    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"]
Oct 30 21:51:01.887301 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:01.886635    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy" containerID="cri-o://65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b" gracePeriod=30
Oct 30 21:51:04.613615 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:04.613577    5786 patch_prober.go:29] interesting pod/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:51:04.614004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:04.613635    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:51:05.582003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.581948    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:51:05.603567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.603546    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:51:05.666703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666682    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-token\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-token\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666714    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-podinfo\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666758    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-kube-api-access-hhxp6\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-kube-api-access-hhxp6\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666793    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-token\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-token\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666845    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee854f35-35ea-4780-aaa6-2421b4026be3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee854f35-35ea-4780-aaa6-2421b4026be3-istiod-ca-cert\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666877    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-envoy\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666935    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-data\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666963    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-credential-socket\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.666998    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-envoy\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667052 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667030    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istiod-ca-cert\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667125    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-data\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667154    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-socket\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667181    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-credential-socket\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667288    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-socket\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667353    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-certs\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667552    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-kube-api-access-56r2z\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-kube-api-access-56r2z\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667608 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667599    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-certs\") pod \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\" (UID: \"aa02a459-0f58-4ccd-ba60-3b55c5d0e121\") "
Oct 30 21:51:05.667978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667638    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-podinfo\") pod \"ee854f35-35ea-4780-aaa6-2421b4026be3\" (UID: \"ee854f35-35ea-4780-aaa6-2421b4026be3\") "
Oct 30 21:51:05.667978 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.667702    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee854f35-35ea-4780-aaa6-2421b4026be3-istiod-ca-cert" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:05.668098 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668051    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-credential-socket" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668180    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee854f35-35ea-4780-aaa6-2421b4026be3-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668217    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668511    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-credential-socket" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668592    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-data" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668661    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-data" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668721    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-certs" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668851    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-socket" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668931    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-socket" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668991    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-certs" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.668999    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istiod-ca-cert" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:05.669854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.669619    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-token" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:05.669854 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.669721    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-kube-api-access-hhxp6" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "kube-api-access-hhxp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:05.669976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.669851    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-envoy" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.669976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.669930    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-podinfo" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:05.670091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.670056    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-token" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:05.670705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.670675    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-podinfo" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:05.670807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.670752    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-envoy" pod "aa02a459-0f58-4ccd-ba60-3b55c5d0e121" (UID: "aa02a459-0f58-4ccd-ba60-3b55c5d0e121"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:05.671005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.670984    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-kube-api-access-56r2z" pod "ee854f35-35ea-4780-aaa6-2421b4026be3" (UID: "ee854f35-35ea-4780-aaa6-2421b4026be3"). InnerVolumeSpecName "kube-api-access-56r2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:05.769307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769276    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769302    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769315    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769328    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769340    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769351    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769362    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56r2z\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-kube-api-access-56r2z\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769374    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769386    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769397    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769409    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769423    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhxp6\" (UniqueName: \"kubernetes.io/projected/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-kube-api-access-hhxp6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769435    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769446    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769458    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee854f35-35ea-4780-aaa6-2421b4026be3-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:05.769520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:05.769469    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa02a459-0f58-4ccd-ba60-3b55c5d0e121-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:06.352033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.352000    5786 generic.go:356] "Generic (PLEG): container finished" podID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerID="4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e" exitCode=0
Oct 30 21:51:06.352182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.352124    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"
Oct 30 21:51:06.353702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.353679    5786 generic.go:356] "Generic (PLEG): container finished" podID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerID="7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201" exitCode=0
Oct 30 21:51:06.353821 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.353800    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"
Oct 30 21:51:06.366221 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.366180    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" event={"ID":"aa02a459-0f58-4ccd-ba60-3b55c5d0e121","Type":"ContainerDied","Data":"4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e"}
Oct 30 21:51:06.366328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.366227    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh" event={"ID":"aa02a459-0f58-4ccd-ba60-3b55c5d0e121","Type":"ContainerDied","Data":"2d20244139a41b38b364881ed416fcc3efbfaf5872a2ca2e5af62d373b0543f7"}
Oct 30 21:51:06.366328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.366239    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" event={"ID":"ee854f35-35ea-4780-aaa6-2421b4026be3","Type":"ContainerDied","Data":"7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201"}
Oct 30 21:51:06.366328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.366248    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm" event={"ID":"ee854f35-35ea-4780-aaa6-2421b4026be3","Type":"ContainerDied","Data":"0f6aa6d36aa24c1fa447261725ea230d68b54ed6919700e678b2ed857e3197e1"}
Oct 30 21:51:06.366328 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.366262    5786 scope.go:122] "RemoveContainer" containerID="4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e"
Oct 30 21:51:06.376171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.376153    5786 scope.go:122] "RemoveContainer" containerID="4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e"
Oct 30 21:51:06.376500 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:51:06.376476    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e\": container with ID starting with 4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e not found: ID does not exist" containerID="4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e"
Oct 30 21:51:06.376586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.376510    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e"} err="failed to get container status \"4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e\": rpc error: code = NotFound desc = could not find container \"4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e\": container with ID starting with 4289c70b9dd40753068be221d69d05d69cf5812274a654e43618d03185c9fc7e not found: ID does not exist"
Oct 30 21:51:06.376586 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.376532    5786 scope.go:122] "RemoveContainer" containerID="7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201"
Oct 30 21:51:06.388981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.388952    5786 scope.go:122] "RemoveContainer" containerID="7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201"
Oct 30 21:51:06.389342 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:51:06.389320    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201\": container with ID starting with 7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201 not found: ID does not exist" containerID="7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201"
Oct 30 21:51:06.389443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.389348    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201"} err="failed to get container status \"7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201\": rpc error: code = NotFound desc = could not find container \"7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201\": container with ID starting with 7d1b207e338ffab2c86504b082920d7a61bf498fb091ae9be9f3a6bd7388f201 not found: ID does not exist"
Oct 30 21:51:06.514763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.514735    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"]
Oct 30 21:51:06.564025 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.563998    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-kj9gm"]
Oct 30 21:51:06.633534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.633444    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"]
Oct 30 21:51:06.636844 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:06.636818    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-be9451-openshift-default-565fb88cd9-jxjqh"]
Oct 30 21:51:07.141402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.141379    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:51:07.286477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286444    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-socket\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.286665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286516    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-envoy\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.286665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286587    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-certs\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.286665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286624    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-token\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-token\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.286839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286686    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-kube-api-access-snbpw\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-kube-api-access-snbpw\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.286839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286710    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-socket" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:07.286839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286758    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-podinfo\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.286839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286799    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istiod-ca-cert\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.287063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286909    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-data\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.287063 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.286941    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-credential-socket\") pod \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\" (UID: \"caa95468-9ab2-4c12-82b9-a9a8793ee3ed\") "
Oct 30 21:51:07.287545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.287269    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-certs" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:07.287545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.287438    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.287545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.287461    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.287545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.287455    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-data" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:07.287545 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.287515    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-credential-socket" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:07.288071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.287571    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istiod-ca-cert" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:07.289580 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.289557    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-token" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:07.289666 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.289624    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-envoy" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:07.290292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.290273    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-podinfo" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:07.290379 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.290363    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-kube-api-access-snbpw" pod "caa95468-9ab2-4c12-82b9-a9a8793ee3ed" (UID: "caa95468-9ab2-4c12-82b9-a9a8793ee3ed"). InnerVolumeSpecName "kube-api-access-snbpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:07.361861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.361837    5786 generic.go:356] "Generic (PLEG): container finished" podID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerID="65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b" exitCode=0
Oct 30 21:51:07.361954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.361877    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"
Oct 30 21:51:07.361954 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.361918    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" event={"ID":"caa95468-9ab2-4c12-82b9-a9a8793ee3ed","Type":"ContainerDied","Data":"65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b"}
Oct 30 21:51:07.362047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.361965    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj" event={"ID":"caa95468-9ab2-4c12-82b9-a9a8793ee3ed","Type":"ContainerDied","Data":"d3cbcaad12fe1d6d5b0b401373c2bb7dbf0615f45370d1519c4d837eb0acd9ad"}
Oct 30 21:51:07.362047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.361988    5786 scope.go:122] "RemoveContainer" containerID="65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b"
Oct 30 21:51:07.376574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.376550    5786 scope.go:122] "RemoveContainer" containerID="65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b"
Oct 30 21:51:07.376849 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:51:07.376826    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b\": container with ID starting with 65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b not found: ID does not exist" containerID="65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b"
Oct 30 21:51:07.376906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.376859    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b"} err="failed to get container status \"65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b\": rpc error: code = NotFound desc = could not find container \"65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b\": container with ID starting with 65525e88e718d149a52eeebf5671798e717060f8f71b7dee86ab14965ad8e15b not found: ID does not exist"
Oct 30 21:51:07.388508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.388489    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.388508 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.388509    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.388620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.388518    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.388620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.388525    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.388620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.388533    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snbpw\" (UniqueName: \"kubernetes.io/projected/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-kube-api-access-snbpw\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.388620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.388542    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.388620 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.388551    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caa95468-9ab2-4c12-82b9-a9a8793ee3ed-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.525781 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.525754    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"]
Oct 30 21:51:07.530587 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:07.530566    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-cp7fj"]
Oct 30 21:51:08.358661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:08.358632    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" path="/var/lib/kubelet/pods/aa02a459-0f58-4ccd-ba60-3b55c5d0e121/volumes"
Oct 30 21:51:08.359071 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:08.359058    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" path="/var/lib/kubelet/pods/caa95468-9ab2-4c12-82b9-a9a8793ee3ed/volumes"
Oct 30 21:51:08.359497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:08.359484    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" path="/var/lib/kubelet/pods/ee854f35-35ea-4780-aaa6-2421b4026be3/volumes"
Oct 30 21:51:16.835881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:16.835853    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"]
Oct 30 21:51:16.836375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:16.836043    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" containerID="cri-o://7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da" gracePeriod=30
Oct 30 21:51:17.018908 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:17.018873    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"]
Oct 30 21:51:17.019247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:17.019228    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"]
Oct 30 21:51:17.019665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:17.019635    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" containerID="cri-o://26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee" gracePeriod=30
Oct 30 21:51:17.020097 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:17.020069    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy" containerID="cri-o://be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84" gracePeriod=30
Oct 30 21:51:17.496491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:17.496458    5786 patch_prober.go:29] interesting pod/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:51:17.496661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:17.496520    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:51:20.677297 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:20.677264    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qs95c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:51:20.677668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:20.677322    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:51:22.187842 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.187811    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:51:22.280189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.280165    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:51:22.303765 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.303745    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:51:22.320030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320001    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-socket\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320042    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-podinfo\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320077    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-token\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-token\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320122    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-credential-socket\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320166    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-data\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320271    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-socket" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.320359 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320281    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istiod-ca-cert\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320469    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-credential-socket" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.320527 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320477    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-certs\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320482    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-data" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.320644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320559    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-kube-api-access-s4w6t\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-kube-api-access-s4w6t\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320602    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-envoy\") pod \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\" (UID: \"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4\") "
Oct 30 21:51:22.320791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320644    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-certs" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.320791 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.320657    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istiod-ca-cert" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:22.321251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.321197    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.321374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.321256    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.321374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.321272    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.321374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.321285    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.321374 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.321300    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.322364 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.322344    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-token" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:22.322470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.322366    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-podinfo" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:22.322555 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.322540    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-kube-api-access-s4w6t" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "kube-api-access-s4w6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:22.322606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.322588    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-envoy" pod "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" (UID: "a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.353566 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:51:22.353538    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:51:22.422171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422144    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-credential-socket\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.422171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422175    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-podinfo\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.422422 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422260    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istiod-ca-cert\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.422422 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422307    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-kube-api-access-hv92b\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-kube-api-access-hv92b\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.422422 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422337    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-socket\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.422422 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422380    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-envoy\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.422422 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422409    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-data\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422458    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-socket\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422467    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-credential-socket" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422488    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-credential-socket\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422550    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-token\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-token\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422580    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-envoy\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422631    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istiod-ca-cert\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422656    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-kube-api-access-6kd9f\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-kube-api-access-6kd9f\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422670    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istiod-ca-cert" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:22.422711 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422700    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-certs\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422738    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-certs\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422764    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-token\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-token\") pod \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\" (UID: \"84691101-48fe-4c5f-ae64-f6faa4bbae0c\") "
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422819    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-data\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422850    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-podinfo\") pod \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\" (UID: \"a988c4e4-539d-4c8d-bc65-bd3298b39ee6\") "
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.422922    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-credential-socket" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423082    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-socket" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423314    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423335    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423351    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423354    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-data" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423364    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423406    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.423428 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423425    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4w6t\" (UniqueName: \"kubernetes.io/projected/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-kube-api-access-s4w6t\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.424118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423443    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.424118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423458    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.424118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423729    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-socket" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.424118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.423905    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-certs" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.424118 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.424067    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-certs" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.424794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.424561    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-data" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.425268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.425233    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istiod-ca-cert" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:22.425499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.425431    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-envoy" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.425644 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.425621    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-podinfo" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:22.425895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.425872    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-token" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:22.425895 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.425889    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-podinfo" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:22.426594 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.426553    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-kube-api-access-hv92b" pod "a988c4e4-539d-4c8d-bc65-bd3298b39ee6" (UID: "a988c4e4-539d-4c8d-bc65-bd3298b39ee6"). InnerVolumeSpecName "kube-api-access-hv92b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:22.426690 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.426651    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-token" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:22.427629 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.427602    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-kube-api-access-6kd9f" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "kube-api-access-6kd9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:22.427783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.427769    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-envoy" pod "84691101-48fe-4c5f-ae64-f6faa4bbae0c" (UID: "84691101-48fe-4c5f-ae64-f6faa4bbae0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:22.457826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.457807    5786 generic.go:356] "Generic (PLEG): container finished" podID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerID="7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da" exitCode=0
Oct 30 21:51:22.457965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.457866    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"
Oct 30 21:51:22.457965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.457883    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" event={"ID":"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4","Type":"ContainerDied","Data":"7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da"}
Oct 30 21:51:22.457965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.457940    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl" event={"ID":"a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4","Type":"ContainerDied","Data":"02941f20da455a1c7c963a485ca3ad3e473efb2f40aaf40731d41f07ffdbeb26"}
Oct 30 21:51:22.458093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.457972    5786 scope.go:122] "RemoveContainer" containerID="7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da"
Oct 30 21:51:22.459493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.459473    5786 generic.go:356] "Generic (PLEG): container finished" podID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerID="26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee" exitCode=0
Oct 30 21:51:22.459609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.459529    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"
Oct 30 21:51:22.459609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.459569    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" event={"ID":"84691101-48fe-4c5f-ae64-f6faa4bbae0c","Type":"ContainerDied","Data":"26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee"}
Oct 30 21:51:22.459720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.459617    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c" event={"ID":"84691101-48fe-4c5f-ae64-f6faa4bbae0c","Type":"ContainerDied","Data":"56c3efea7486b71f34a4c73516d83f1c9a74759b49c3a3219873a1e3bc2088f6"}
Oct 30 21:51:22.461226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.461191    5786 generic.go:356] "Generic (PLEG): container finished" podID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerID="be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84" exitCode=0
Oct 30 21:51:22.461335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.461248    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" event={"ID":"a988c4e4-539d-4c8d-bc65-bd3298b39ee6","Type":"ContainerDied","Data":"be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84"}
Oct 30 21:51:22.461335 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.461282    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"
Oct 30 21:51:22.461457 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.461289    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54" event={"ID":"a988c4e4-539d-4c8d-bc65-bd3298b39ee6","Type":"ContainerDied","Data":"f1dca00d79b587d529cab8f177ce02b13ce3673ff38b4fbb490c86666d1bfd66"}
Oct 30 21:51:22.468730 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.468710    5786 scope.go:122] "RemoveContainer" containerID="7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da"
Oct 30 21:51:22.469029 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:51:22.469005    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da\": container with ID starting with 7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da not found: ID does not exist" containerID="7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da"
Oct 30 21:51:22.469101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.469037    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da"} err="failed to get container status \"7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da\": rpc error: code = NotFound desc = could not find container \"7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da\": container with ID starting with 7b0704c3d16a1bc790f5d0c65906da96732673098de43a17aeab2a578ef8a1da not found: ID does not exist"
Oct 30 21:51:22.469101 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.469055    5786 scope.go:122] "RemoveContainer" containerID="26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee"
Oct 30 21:51:22.481799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.481779    5786 scope.go:122] "RemoveContainer" containerID="26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee"
Oct 30 21:51:22.482087 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:51:22.482070    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee\": container with ID starting with 26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee not found: ID does not exist" containerID="26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee"
Oct 30 21:51:22.482164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.482092    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee"} err="failed to get container status \"26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee\": rpc error: code = NotFound desc = could not find container \"26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee\": container with ID starting with 26d2d3d3bbfd7c6d586fc71c08711d08e4932a17a7821b19420c317a2bda2fee not found: ID does not exist"
Oct 30 21:51:22.482164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.482109    5786 scope.go:122] "RemoveContainer" containerID="be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84"
Oct 30 21:51:22.490712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.490690    5786 scope.go:122] "RemoveContainer" containerID="be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84"
Oct 30 21:51:22.491016 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:51:22.490993    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84\": container with ID starting with be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84 not found: ID does not exist" containerID="be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84"
Oct 30 21:51:22.491081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.491023    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84"} err="failed to get container status \"be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84\": rpc error: code = NotFound desc = could not find container \"be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84\": container with ID starting with be65aa7f0f9ac383394b12075785c4f6dd1b4967b225870944a065ce9e221d84 not found: ID does not exist"
Oct 30 21:51:22.524516 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524492    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv92b\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-kube-api-access-hv92b\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524521    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524539    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524555    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524569    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524584    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524599    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524633 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524616    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kd9f\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-kube-api-access-6kd9f\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524645    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84691101-48fe-4c5f-ae64-f6faa4bbae0c-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524667    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524682    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524696    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524709    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a988c4e4-539d-4c8d-bc65-bd3298b39ee6-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.524906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.524722    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84691101-48fe-4c5f-ae64-f6faa4bbae0c-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:22.748877 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.748844    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"]
Oct 30 21:51:22.756454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.756423    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-8ce31a-openshift-default-bc8b5cfb5-s5psl"]
Oct 30 21:51:22.863340 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.863308    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"]
Oct 30 21:51:22.866406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.866383    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-nsg54"]
Oct 30 21:51:22.974146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.974114    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"]
Oct 30 21:51:22.979039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:22.979017    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qs95c"]
Oct 30 21:51:24.358920 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:24.358889    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" path="/var/lib/kubelet/pods/84691101-48fe-4c5f-ae64-f6faa4bbae0c/volumes"
Oct 30 21:51:24.359509 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:24.359489    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" path="/var/lib/kubelet/pods/a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4/volumes"
Oct 30 21:51:24.359944 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:24.359923    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" path="/var/lib/kubelet/pods/a988c4e4-539d-4c8d-bc65-bd3298b39ee6/volumes"
Oct 30 21:51:27.223121 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:27.223091    5786 scope.go:122] "RemoveContainer" containerID="95f753cfe39ee732efed5d0689fb53843d3cad170b8d3dc1d7701fc685b8d701"
Oct 30 21:51:27.232190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:51:27.232169    5786 scope.go:122] "RemoveContainer" containerID="83bee20273b25c6023d4a81f59cb4d7c7e2e824bf0a6bac381876ccfdac8ea9a"
Oct 30 21:51:52.352470 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:51:52.352379    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:52:03.050668 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:03.050638    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"]
Oct 30 21:52:03.099779 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:03.099753    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-309c89c5-d7rx7"]
Oct 30 21:52:04.366327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:04.366287    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a186ff7-34e9-4338-829a-a951fd33d5f4" path="/var/lib/kubelet/pods/3a186ff7-34e9-4338-829a-a951fd33d5f4/volumes"
Oct 30 21:52:17.918270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918243    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"]
Oct 30 21:52:17.918721 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918702    5786 cpu_manager.go:414] "containerMap: removing container" podUID="16e6d78c-7d7b-4c62-be32-5808590e1345" containerName="payload-pre-processing"
Oct 30 21:52:17.918721 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918715    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16e6d78c-7d7b-4c62-be32-5808590e1345" containerName="payload-pre-processing"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918724    5786 cpu_manager.go:414] "containerMap: removing container" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918729    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918738    5786 cpu_manager.go:414] "containerMap: removing container" podUID="b9679d46-0e10-4523-af31-7362a4871af7" containerName="payload-processing"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918742    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9679d46-0e10-4523-af31-7362a4871af7" containerName="payload-processing"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918765    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918770    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918777    5786 cpu_manager.go:414] "containerMap: removing container" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918782    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918788    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918792    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918798    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy"
Oct 30 21:52:17.918798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918803    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918808    5786 cpu_manager.go:414] "containerMap: removing container" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918812    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918824    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7bc433cb-99ff-4a5e-a250-00dfa44ed156" containerName="revoke-keys"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918828    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bc433cb-99ff-4a5e-a250-00dfa44ed156" containerName="revoke-keys"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918834    5786 cpu_manager.go:414] "containerMap: removing container" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918839    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918895    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa02a459-0f58-4ccd-ba60-3b55c5d0e121" containerName="istio-proxy"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918902    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee854f35-35ea-4780-aaa6-2421b4026be3" containerName="istio-proxy"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918907    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a988c4e4-539d-4c8d-bc65-bd3298b39ee6" containerName="istio-proxy"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918913    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="caa95468-9ab2-4c12-82b9-a9a8793ee3ed" containerName="istio-proxy"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918919    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bc433cb-99ff-4a5e-a250-00dfa44ed156" containerName="revoke-keys"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918925    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="61f2966e-e0d9-44b0-8ac5-69d7a678a0d8" containerName="main"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918933    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="16e6d78c-7d7b-4c62-be32-5808590e1345" containerName="payload-pre-processing"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918938    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="84691101-48fe-4c5f-ae64-f6faa4bbae0c" containerName="istio-proxy"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918944    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9679d46-0e10-4523-af31-7362a4871af7" containerName="payload-processing"
Oct 30 21:52:17.919189 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.918953    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1c4c0d7-3d6b-4123-8892-7a1a5e9b75e4" containerName="istio-proxy"
Oct 30 21:52:17.922141 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:17.922123    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.021512 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.021479    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.021681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.021520    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.021681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.021542    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdnw4\" (UniqueName: \"kubernetes.io/projected/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-kube-api-access-sdnw4\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.026310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.026289    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:52:18.026409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.026289    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:52:18.028184 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.028166    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"]
Oct 30 21:52:18.122722 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.122697    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.122848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.122728    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.122848 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.122747    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdnw4\" (UniqueName: \"kubernetes.io/projected/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-kube-api-access-sdnw4\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.123302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.123284    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.125119 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.125098    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.178796 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.178740    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdnw4\" (UniqueName: \"kubernetes.io/projected/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-kube-api-access-sdnw4\") pod \"maas-api-revoke-keys-e2e-mt-3ab88248-qckm9\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.235468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.235446    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:18.606680 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.606657    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"]
Oct 30 21:52:18.609458 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:52:18.609431    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e8b1eaf_a924_49ff_9942_7c19fcbac96e.slice/crio-53e30fc361eab10cdf51bd9de707f77ba7557b3ebc21096272f0dec88f5cd8a7 WatchSource:0}: Error finding container 53e30fc361eab10cdf51bd9de707f77ba7557b3ebc21096272f0dec88f5cd8a7: Status 404 returned error can't find the container with id 53e30fc361eab10cdf51bd9de707f77ba7557b3ebc21096272f0dec88f5cd8a7
Oct 30 21:52:18.773943 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:18.773915    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9" event={"ID":"5e8b1eaf-a924-49ff-9942-7c19fcbac96e","Type":"ContainerStarted","Data":"53e30fc361eab10cdf51bd9de707f77ba7557b3ebc21096272f0dec88f5cd8a7"}
Oct 30 21:52:19.781451 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:19.781368    5786 generic.go:356] "Generic (PLEG): container finished" podID="5e8b1eaf-a924-49ff-9942-7c19fcbac96e" containerID="02784a96f9bec834d841146a02a1d39a24f3145420c0d5db2d8149d96e0b7c37" exitCode=0
Oct 30 21:52:19.781830 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:19.781453    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9" event={"ID":"5e8b1eaf-a924-49ff-9942-7c19fcbac96e","Type":"ContainerDied","Data":"02784a96f9bec834d841146a02a1d39a24f3145420c0d5db2d8149d96e0b7c37"}
Oct 30 21:52:20.906075 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:20.906050    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:21.046325 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.046260    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-kube-api-access-sdnw4\" (UniqueName: \"kubernetes.io/projected/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-kube-api-access-sdnw4\") pod \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") "
Oct 30 21:52:21.046463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.046352    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-openshift-ca\") pod \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") "
Oct 30 21:52:21.046463 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.046424    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-xks-ca\") pod \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\" (UID: \"5e8b1eaf-a924-49ff-9942-7c19fcbac96e\") "
Oct 30 21:52:21.046662 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.046638    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-openshift-ca" pod "5e8b1eaf-a924-49ff-9942-7c19fcbac96e" (UID: "5e8b1eaf-a924-49ff-9942-7c19fcbac96e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:21.046898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.046882    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.048499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.048476    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-kube-api-access-sdnw4" pod "5e8b1eaf-a924-49ff-9942-7c19fcbac96e" (UID: "5e8b1eaf-a924-49ff-9942-7c19fcbac96e"). InnerVolumeSpecName "kube-api-access-sdnw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:21.048609 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.048576    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-xks-ca" pod "5e8b1eaf-a924-49ff-9942-7c19fcbac96e" (UID: "5e8b1eaf-a924-49ff-9942-7c19fcbac96e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:52:21.147635 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.147614    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdnw4\" (UniqueName: \"kubernetes.io/projected/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-kube-api-access-sdnw4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.147635 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.147636    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e8b1eaf-a924-49ff-9942-7c19fcbac96e-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.793763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.793738    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"
Oct 30 21:52:21.793763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.793760    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9" event={"ID":"5e8b1eaf-a924-49ff-9942-7c19fcbac96e","Type":"ContainerDied","Data":"53e30fc361eab10cdf51bd9de707f77ba7557b3ebc21096272f0dec88f5cd8a7"}
Oct 30 21:52:21.793965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:21.793798    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53e30fc361eab10cdf51bd9de707f77ba7557b3ebc21096272f0dec88f5cd8a7"
Oct 30 21:52:22.353926 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:52:22.353892    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:52:27.305320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:27.305291    5786 scope.go:122] "RemoveContainer" containerID="56655509c7c9752fe0c1453e7a7ab491ce3a0233367c6b4395b80cd57bd5cd82"
Oct 30 21:52:27.314307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:27.314288    5786 scope.go:122] "RemoveContainer" containerID="f4b3779e9ebb7014cf933981b1760943d50ae20636d8d68c033e83434e437f6d"
Oct 30 21:52:27.322173 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:27.322158    5786 scope.go:122] "RemoveContainer" containerID="734ff0083b13d63d58f07055a7c2f50f1767ad519def984209ced8b464f08e40"
Oct 30 21:52:27.329930 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:27.329896    5786 scope.go:122] "RemoveContainer" containerID="6f5d6ddd2b28091ed83a38868949b4252348f927478030051a7a0ce41d111851"
Oct 30 21:52:38.227368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:38.227336    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"]
Oct 30 21:52:38.276190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:38.276162    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ab88248-qckm9"]
Oct 30 21:52:38.357674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:52:38.357648    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e8b1eaf-a924-49ff-9942-7c19fcbac96e" path="/var/lib/kubelet/pods/5e8b1eaf-a924-49ff-9942-7c19fcbac96e/volumes"
Oct 30 21:52:52.361173 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:52:52.361137    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:53:22.359431 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:53:22.359351    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:53:27.375231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:27.375182    5786 scope.go:122] "RemoveContainer" containerID="1f17a36ddf2021b3e891d22139a7017a4249c9e7632f163c6c5579608fae8efe"
Oct 30 21:53:27.385429 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:27.385407    5786 scope.go:122] "RemoveContainer" containerID="38aa8429f90beb03f2ff79505c94d5e24cffaaa20123540b9a6c0f0d8f692684"
Oct 30 21:53:28.677632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.677598    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"]
Oct 30 21:53:28.678110 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.678096    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5e8b1eaf-a924-49ff-9942-7c19fcbac96e" containerName="revoke-keys"
Oct 30 21:53:28.678177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.678112    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e8b1eaf-a924-49ff-9942-7c19fcbac96e" containerName="revoke-keys"
Oct 30 21:53:28.678250 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.678224    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e8b1eaf-a924-49ff-9942-7c19fcbac96e" containerName="revoke-keys"
Oct 30 21:53:28.682968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.682953    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.766059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.766031    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.766260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.766071    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2sx4\" (UniqueName: \"kubernetes.io/projected/bd16c744-8e51-427c-8ddb-b6fc9508f361-kube-api-access-m2sx4\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.766260 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.766193    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.784309 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.784281    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:53:28.784455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.784319    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:53:28.786380 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.786360    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"]
Oct 30 21:53:28.866937 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.866917    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.867064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.866945    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2sx4\" (UniqueName: \"kubernetes.io/projected/bd16c744-8e51-427c-8ddb-b6fc9508f361-kube-api-access-m2sx4\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.867064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.866987    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.867670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.867651    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.869461 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.869443    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.922864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.922840    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2sx4\" (UniqueName: \"kubernetes.io/projected/bd16c744-8e51-427c-8ddb-b6fc9508f361-kube-api-access-m2sx4\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:28.994922 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:28.994875    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:29.055948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:29.048744    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"]
Oct 30 21:53:29.055948 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:29.053824    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6d54aed-q6k8k"]
Oct 30 21:53:29.058159 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:29.056704    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"]
Oct 30 21:53:29.061767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:29.059948    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8ceaeccc-m7ftx"]
Oct 30 21:53:29.366193 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:29.366170    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"]
Oct 30 21:53:29.368900 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:53:29.368872    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd16c744_8e51_427c_8ddb_b6fc9508f361.slice/crio-c342b73d91a4521d8f2eb71e6d97b6defc1e7e40039b21492bc43081118cf914 WatchSource:0}: Error finding container c342b73d91a4521d8f2eb71e6d97b6defc1e7e40039b21492bc43081118cf914: Status 404 returned error can't find the container with id c342b73d91a4521d8f2eb71e6d97b6defc1e7e40039b21492bc43081118cf914
Oct 30 21:53:30.166123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:30.166088    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl" event={"ID":"bd16c744-8e51-427c-8ddb-b6fc9508f361","Type":"ContainerStarted","Data":"5d756a57f2c0aa964675eaed7a03aca84218403b7b0b63ab2d900fc27a0de4e8"}
Oct 30 21:53:30.166625 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:30.166153    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl" event={"ID":"bd16c744-8e51-427c-8ddb-b6fc9508f361","Type":"ContainerStarted","Data":"c342b73d91a4521d8f2eb71e6d97b6defc1e7e40039b21492bc43081118cf914"}
Oct 30 21:53:30.280893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:30.280843    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl" podStartSLOduration=2.2808276210000002 podStartE2EDuration="2.280827621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:30.280350298 +0000 UTC m=+2466.151469418" watchObservedRunningTime="2026-10-30 21:53:30.280827621 +0000 UTC m=+2466.151946739"
Oct 30 21:53:30.358864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:30.358830    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95a4b51d-7533-4db9-9928-d8dfca8752ee" path="/var/lib/kubelet/pods/95a4b51d-7533-4db9-9928-d8dfca8752ee/volumes"
Oct 30 21:53:30.359268 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:30.359254    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3eeb03c-de60-4d0f-af50-6217c7c02d48" path="/var/lib/kubelet/pods/b3eeb03c-de60-4d0f-af50-6217c7c02d48/volumes"
Oct 30 21:53:31.172518 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:31.172486    5786 generic.go:356] "Generic (PLEG): container finished" podID="bd16c744-8e51-427c-8ddb-b6fc9508f361" containerID="5d756a57f2c0aa964675eaed7a03aca84218403b7b0b63ab2d900fc27a0de4e8" exitCode=0
Oct 30 21:53:31.172890 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:31.172566    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl" event={"ID":"bd16c744-8e51-427c-8ddb-b6fc9508f361","Type":"ContainerDied","Data":"5d756a57f2c0aa964675eaed7a03aca84218403b7b0b63ab2d900fc27a0de4e8"}
Oct 30 21:53:32.317718 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.317696    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:32.397866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.397843    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-openshift-ca\") pod \"bd16c744-8e51-427c-8ddb-b6fc9508f361\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") "
Oct 30 21:53:32.398006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.397882    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd16c744-8e51-427c-8ddb-b6fc9508f361-kube-api-access-m2sx4\" (UniqueName: \"kubernetes.io/projected/bd16c744-8e51-427c-8ddb-b6fc9508f361-kube-api-access-m2sx4\") pod \"bd16c744-8e51-427c-8ddb-b6fc9508f361\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") "
Oct 30 21:53:32.398006 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.398001    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-xks-ca\") pod \"bd16c744-8e51-427c-8ddb-b6fc9508f361\" (UID: \"bd16c744-8e51-427c-8ddb-b6fc9508f361\") "
Oct 30 21:53:32.398245 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.398194    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-openshift-ca" pod "bd16c744-8e51-427c-8ddb-b6fc9508f361" (UID: "bd16c744-8e51-427c-8ddb-b6fc9508f361"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:32.400066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.400044    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-xks-ca" pod "bd16c744-8e51-427c-8ddb-b6fc9508f361" (UID: "bd16c744-8e51-427c-8ddb-b6fc9508f361"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:32.400194 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.400180    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd16c744-8e51-427c-8ddb-b6fc9508f361-kube-api-access-m2sx4" pod "bd16c744-8e51-427c-8ddb-b6fc9508f361" (UID: "bd16c744-8e51-427c-8ddb-b6fc9508f361"). InnerVolumeSpecName "kube-api-access-m2sx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:32.498797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.498744    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:32.498797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.498767    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2sx4\" (UniqueName: \"kubernetes.io/projected/bd16c744-8e51-427c-8ddb-b6fc9508f361-kube-api-access-m2sx4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:32.498797 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:32.498778    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd16c744-8e51-427c-8ddb-b6fc9508f361-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:33.184190 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:33.184162    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"
Oct 30 21:53:33.184366 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:33.184162    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl" event={"ID":"bd16c744-8e51-427c-8ddb-b6fc9508f361","Type":"ContainerDied","Data":"c342b73d91a4521d8f2eb71e6d97b6defc1e7e40039b21492bc43081118cf914"}
Oct 30 21:53:33.184366 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:33.184301    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c342b73d91a4521d8f2eb71e6d97b6defc1e7e40039b21492bc43081118cf914"
Oct 30 21:53:40.145721 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:40.145684    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"]
Oct 30 21:53:40.257237 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:40.257179    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"]
Oct 30 21:53:45.147034 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.146986    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs" podUID="977c46a1-044a-4545-8181-58c5ba1b0a18" containerName="payload-processing" containerID="cri-o://bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3" gracePeriod=25
Oct 30 21:53:45.257610 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.257543    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b" podUID="91bd3b74-19bf-4b72-b892-dc85d1dec133" containerName="payload-pre-processing" containerID="cri-o://5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e" gracePeriod=25
Oct 30 21:53:45.396983 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.396960    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:53:45.500177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.500157    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:53:45.515140 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.515113    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/977c46a1-044a-4545-8181-58c5ba1b0a18-kube-api-access-cf7wb\" (UniqueName: \"kubernetes.io/projected/977c46a1-044a-4545-8181-58c5ba1b0a18-kube-api-access-cf7wb\") pod \"977c46a1-044a-4545-8181-58c5ba1b0a18\" (UID: \"977c46a1-044a-4545-8181-58c5ba1b0a18\") "
Oct 30 21:53:45.515298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.515280    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/977c46a1-044a-4545-8181-58c5ba1b0a18-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977c46a1-044a-4545-8181-58c5ba1b0a18-plugins-config-volume\") pod \"977c46a1-044a-4545-8181-58c5ba1b0a18\" (UID: \"977c46a1-044a-4545-8181-58c5ba1b0a18\") "
Oct 30 21:53:45.515678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.515644    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/977c46a1-044a-4545-8181-58c5ba1b0a18-plugins-config-volume" pod "977c46a1-044a-4545-8181-58c5ba1b0a18" (UID: "977c46a1-044a-4545-8181-58c5ba1b0a18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:45.517430 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.517408    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/977c46a1-044a-4545-8181-58c5ba1b0a18-kube-api-access-cf7wb" pod "977c46a1-044a-4545-8181-58c5ba1b0a18" (UID: "977c46a1-044a-4545-8181-58c5ba1b0a18"). InnerVolumeSpecName "kube-api-access-cf7wb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:45.616822 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.616803    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91bd3b74-19bf-4b72-b892-dc85d1dec133-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91bd3b74-19bf-4b72-b892-dc85d1dec133-plugins-config-volume\") pod \"91bd3b74-19bf-4b72-b892-dc85d1dec133\" (UID: \"91bd3b74-19bf-4b72-b892-dc85d1dec133\") "
Oct 30 21:53:45.616932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.616854    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91bd3b74-19bf-4b72-b892-dc85d1dec133-kube-api-access-vcgcm\" (UniqueName: \"kubernetes.io/projected/91bd3b74-19bf-4b72-b892-dc85d1dec133-kube-api-access-vcgcm\") pod \"91bd3b74-19bf-4b72-b892-dc85d1dec133\" (UID: \"91bd3b74-19bf-4b72-b892-dc85d1dec133\") "
Oct 30 21:53:45.617125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.617102    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91bd3b74-19bf-4b72-b892-dc85d1dec133-plugins-config-volume" pod "91bd3b74-19bf-4b72-b892-dc85d1dec133" (UID: "91bd3b74-19bf-4b72-b892-dc85d1dec133"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:45.617125 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.617121    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf7wb\" (UniqueName: \"kubernetes.io/projected/977c46a1-044a-4545-8181-58c5ba1b0a18-kube-api-access-cf7wb\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:45.617279 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.617132    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977c46a1-044a-4545-8181-58c5ba1b0a18-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:45.618865 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.618843    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91bd3b74-19bf-4b72-b892-dc85d1dec133-kube-api-access-vcgcm" pod "91bd3b74-19bf-4b72-b892-dc85d1dec133" (UID: "91bd3b74-19bf-4b72-b892-dc85d1dec133"). InnerVolumeSpecName "kube-api-access-vcgcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:45.718676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.718623    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91bd3b74-19bf-4b72-b892-dc85d1dec133-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:45.718676 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:45.718645    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcgcm\" (UniqueName: \"kubernetes.io/projected/91bd3b74-19bf-4b72-b892-dc85d1dec133-kube-api-access-vcgcm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:46.255102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.255076    5786 generic.go:356] "Generic (PLEG): container finished" podID="91bd3b74-19bf-4b72-b892-dc85d1dec133" containerID="5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e" exitCode=0
Oct 30 21:53:46.255488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.255116    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"
Oct 30 21:53:46.255488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.255157    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b" event={"ID":"91bd3b74-19bf-4b72-b892-dc85d1dec133","Type":"ContainerDied","Data":"5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e"}
Oct 30 21:53:46.255488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.255222    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b" event={"ID":"91bd3b74-19bf-4b72-b892-dc85d1dec133","Type":"ContainerDied","Data":"ebca06a6da727fd2bfd789cc628ed5f027a33ac5c1fa6e4b59886fadf57c3488"}
Oct 30 21:53:46.255488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.255250    5786 scope.go:122] "RemoveContainer" containerID="5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e"
Oct 30 21:53:46.256836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.256818    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"
Oct 30 21:53:46.256964 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.256844    5786 generic.go:356] "Generic (PLEG): container finished" podID="977c46a1-044a-4545-8181-58c5ba1b0a18" containerID="bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3" exitCode=0
Oct 30 21:53:46.257039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.256902    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs" event={"ID":"977c46a1-044a-4545-8181-58c5ba1b0a18","Type":"ContainerDied","Data":"bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3"}
Oct 30 21:53:46.257039 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.257021    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs" event={"ID":"977c46a1-044a-4545-8181-58c5ba1b0a18","Type":"ContainerDied","Data":"a9990fc3d8fc5137e3a16d7b5a8b1e27b6bd02dddb247e23e83706a09431f633"}
Oct 30 21:53:46.271083 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.271059    5786 scope.go:122] "RemoveContainer" containerID="5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e"
Oct 30 21:53:46.271517 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:53:46.271487    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e\": container with ID starting with 5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e not found: ID does not exist" containerID="5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e"
Oct 30 21:53:46.271906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.271870    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e"} err="failed to get container status \"5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e\": rpc error: code = NotFound desc = could not find container \"5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e\": container with ID starting with 5ef097e8d689d2f20229a1b810b85ac49764728d574f672ada347283132e326e not found: ID does not exist"
Oct 30 21:53:46.271906 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.271906    5786 scope.go:122] "RemoveContainer" containerID="bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3"
Oct 30 21:53:46.281139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.281124    5786 scope.go:122] "RemoveContainer" containerID="bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3"
Oct 30 21:53:46.281410 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:53:46.281393    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3\": container with ID starting with bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3 not found: ID does not exist" containerID="bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3"
Oct 30 21:53:46.281467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.281418    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3"} err="failed to get container status \"bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3\": rpc error: code = NotFound desc = could not find container \"bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3\": container with ID starting with bd81bffadbad43fd9be9b0822d3c0a6dbc92eefaf6348a3a1354204ef8499ab3 not found: ID does not exist"
Oct 30 21:53:46.416310 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.416290    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"]
Oct 30 21:53:46.465631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.465613    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-be8b73-7b79f8c85c-k7jqs"]
Oct 30 21:53:46.529257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.529230    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"]
Oct 30 21:53:46.533062 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:46.533043    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-be8b73-57fc55c54c-n4v4b"]
Oct 30 21:53:48.357766 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:48.357735    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91bd3b74-19bf-4b72-b892-dc85d1dec133" path="/var/lib/kubelet/pods/91bd3b74-19bf-4b72-b892-dc85d1dec133/volumes"
Oct 30 21:53:48.358144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:48.358078    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="977c46a1-044a-4545-8181-58c5ba1b0a18" path="/var/lib/kubelet/pods/977c46a1-044a-4545-8181-58c5ba1b0a18/volumes"
Oct 30 21:53:52.353096 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:53:52.353063    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:53:55.661468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:55.661429    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"]
Oct 30 21:53:55.710752 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:55.710725    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-be8b73-6j8nl"]
Oct 30 21:53:56.357631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:53:56.357594    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd16c744-8e51-427c-8ddb-b6fc9508f361" path="/var/lib/kubelet/pods/bd16c744-8e51-427c-8ddb-b6fc9508f361/volumes"
Oct 30 21:54:02.892253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:02.892195    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"]
Oct 30 21:54:02.892789 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:02.892418    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy" containerID="cri-o://2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197" gracePeriod=30
Oct 30 21:54:08.142616 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.142590    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:54:08.234332 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234305    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-certs\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.234489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234361    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-data\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.234489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234412    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-credential-socket\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.234489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234430    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-envoy\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.234489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234447    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-socket\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.234489 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234470    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-kube-api-access-klr4r\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-kube-api-access-klr4r\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.234773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234565    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69bebe1c-95dc-4834-b836-bbd28f21f95d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bebe1c-95dc-4834-b836-bbd28f21f95d-istiod-ca-cert\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.234773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234586    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-certs" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:08.234773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234606    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-token\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-token\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.234773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234642    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-podinfo\") pod \"69bebe1c-95dc-4834-b836-bbd28f21f95d\" (UID: \"69bebe1c-95dc-4834-b836-bbd28f21f95d\") "
Oct 30 21:54:08.235003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234639    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-data" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:08.235003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234914    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69bebe1c-95dc-4834-b836-bbd28f21f95d-istiod-ca-cert" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:08.235003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.234743    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-credential-socket" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:08.235176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.235015    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.235176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.235028    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-socket" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:08.235176 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.235035    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.236737 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.236711    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-envoy" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:08.236843 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.236746    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-kube-api-access-klr4r" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "kube-api-access-klr4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:08.236843 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.236766    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-token" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:08.236931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.236912    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-podinfo" pod "69bebe1c-95dc-4834-b836-bbd28f21f95d" (UID: "69bebe1c-95dc-4834-b836-bbd28f21f95d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:08.335859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.335831    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.335859 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.335858    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.336031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.335868    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bebe1c-95dc-4834-b836-bbd28f21f95d-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.336031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.335877    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klr4r\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-kube-api-access-klr4r\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.336031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.335887    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bebe1c-95dc-4834-b836-bbd28f21f95d-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.336031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.335896    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.336031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.335904    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bebe1c-95dc-4834-b836-bbd28f21f95d-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:08.372169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.372144    5786 generic.go:356] "Generic (PLEG): container finished" podID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerID="2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197" exitCode=0
Oct 30 21:54:08.372289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.372188    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"
Oct 30 21:54:08.372289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.372230    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" event={"ID":"69bebe1c-95dc-4834-b836-bbd28f21f95d","Type":"ContainerDied","Data":"2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197"}
Oct 30 21:54:08.372289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.372282    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx" event={"ID":"69bebe1c-95dc-4834-b836-bbd28f21f95d","Type":"ContainerDied","Data":"0c7a7f2f568fa669e2643c25b183e821635dd5ac1342b82fec32a442dfa51cea"}
Oct 30 21:54:08.372409 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.372308    5786 scope.go:122] "RemoveContainer" containerID="2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197"
Oct 30 21:54:08.381470 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.381452    5786 scope.go:122] "RemoveContainer" containerID="5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23"
Oct 30 21:54:08.389553 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.389529    5786 scope.go:122] "RemoveContainer" containerID="2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197"
Oct 30 21:54:08.389835 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:54:08.389816    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197\": container with ID starting with 2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197 not found: ID does not exist" containerID="2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197"
Oct 30 21:54:08.389886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.389844    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197"} err="failed to get container status \"2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197\": rpc error: code = NotFound desc = could not find container \"2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197\": container with ID starting with 2253690d9909d3d9742de79d992dd461de92444f886c26070f6cc5f615fdc197 not found: ID does not exist"
Oct 30 21:54:08.389886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.389861    5786 scope.go:122] "RemoveContainer" containerID="5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23"
Oct 30 21:54:08.390080 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:54:08.390065    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23\": container with ID starting with 5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23 not found: ID does not exist" containerID="5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23"
Oct 30 21:54:08.390123 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.390083    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23"} err="failed to get container status \"5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23\": rpc error: code = NotFound desc = could not find container \"5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23\": container with ID starting with 5a17beb44cf27dc8c369050ea313d7e744d42c607ad5602cca8034d558b35d23 not found: ID does not exist"
Oct 30 21:54:08.536388 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.536323    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"]
Oct 30 21:54:08.539802 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:08.539773    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-be8b73-openshift-default-589b6b8499-wckcx"]
Oct 30 21:54:10.360683 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:10.360645    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" path="/var/lib/kubelet/pods/69bebe1c-95dc-4834-b836-bbd28f21f95d/volumes"
Oct 30 21:54:15.194467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:15.194414    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"]
Oct 30 21:54:15.300872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:15.300831    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"]
Oct 30 21:54:20.195376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.195314    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp" podUID="1a2feebb-1715-45b3-b8e5-6c808b94c262" containerName="payload-processing" containerID="cri-o://5dd672e4281b230e16677f37dde8e025568eb3fca0640c4385821e8402cc88fd" gracePeriod=25
Oct 30 21:54:20.302247 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.302186    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg" podUID="49d006ce-9d62-4a69-bed0-3e2d89522c0b" containerName="payload-pre-processing" containerID="cri-o://85883410d664f62981d4a851a37d99339a1eb93e55eec37291fe836dd6a0fc3c" gracePeriod=25
Oct 30 21:54:20.438357 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.438326    5786 generic.go:356] "Generic (PLEG): container finished" podID="49d006ce-9d62-4a69-bed0-3e2d89522c0b" containerID="85883410d664f62981d4a851a37d99339a1eb93e55eec37291fe836dd6a0fc3c" exitCode=0
Oct 30 21:54:20.438488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.438398    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg" event={"ID":"49d006ce-9d62-4a69-bed0-3e2d89522c0b","Type":"ContainerDied","Data":"85883410d664f62981d4a851a37d99339a1eb93e55eec37291fe836dd6a0fc3c"}
Oct 30 21:54:20.440012 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.439987    5786 generic.go:356] "Generic (PLEG): container finished" podID="1a2feebb-1715-45b3-b8e5-6c808b94c262" containerID="5dd672e4281b230e16677f37dde8e025568eb3fca0640c4385821e8402cc88fd" exitCode=0
Oct 30 21:54:20.440129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.440056    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp" event={"ID":"1a2feebb-1715-45b3-b8e5-6c808b94c262","Type":"ContainerDied","Data":"5dd672e4281b230e16677f37dde8e025568eb3fca0640c4385821e8402cc88fd"}
Oct 30 21:54:20.440129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.440108    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp" event={"ID":"1a2feebb-1715-45b3-b8e5-6c808b94c262","Type":"ContainerDied","Data":"96a57452706245c6db140c99303470546a60f16b1d80856b764e4555abaae202"}
Oct 30 21:54:20.440243 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.440128    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96a57452706245c6db140c99303470546a60f16b1d80856b764e4555abaae202"
Oct 30 21:54:20.453768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.453710    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:54:20.545817 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.545794    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a2feebb-1715-45b3-b8e5-6c808b94c262-kube-api-access-xzgpv\" (UniqueName: \"kubernetes.io/projected/1a2feebb-1715-45b3-b8e5-6c808b94c262-kube-api-access-xzgpv\") pod \"1a2feebb-1715-45b3-b8e5-6c808b94c262\" (UID: \"1a2feebb-1715-45b3-b8e5-6c808b94c262\") "
Oct 30 21:54:20.545953 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.545873    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a2feebb-1715-45b3-b8e5-6c808b94c262-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a2feebb-1715-45b3-b8e5-6c808b94c262-plugins-config-volume\") pod \"1a2feebb-1715-45b3-b8e5-6c808b94c262\" (UID: \"1a2feebb-1715-45b3-b8e5-6c808b94c262\") "
Oct 30 21:54:20.546252 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.546230    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a2feebb-1715-45b3-b8e5-6c808b94c262-plugins-config-volume" pod "1a2feebb-1715-45b3-b8e5-6c808b94c262" (UID: "1a2feebb-1715-45b3-b8e5-6c808b94c262"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:20.547812 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.547791    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a2feebb-1715-45b3-b8e5-6c808b94c262-kube-api-access-xzgpv" pod "1a2feebb-1715-45b3-b8e5-6c808b94c262" (UID: "1a2feebb-1715-45b3-b8e5-6c808b94c262"). InnerVolumeSpecName "kube-api-access-xzgpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:20.558480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.558461    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:54:20.647265 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.647245    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49d006ce-9d62-4a69-bed0-3e2d89522c0b-kube-api-access-65jrs\" (UniqueName: \"kubernetes.io/projected/49d006ce-9d62-4a69-bed0-3e2d89522c0b-kube-api-access-65jrs\") pod \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\" (UID: \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\") "
Oct 30 21:54:20.647372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.647341    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49d006ce-9d62-4a69-bed0-3e2d89522c0b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d006ce-9d62-4a69-bed0-3e2d89522c0b-plugins-config-volume\") pod \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\" (UID: \"49d006ce-9d62-4a69-bed0-3e2d89522c0b\") "
Oct 30 21:54:20.647729 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.647709    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzgpv\" (UniqueName: \"kubernetes.io/projected/1a2feebb-1715-45b3-b8e5-6c808b94c262-kube-api-access-xzgpv\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:20.647975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.647739    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a2feebb-1715-45b3-b8e5-6c808b94c262-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:20.647975 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.647717    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d006ce-9d62-4a69-bed0-3e2d89522c0b-plugins-config-volume" pod "49d006ce-9d62-4a69-bed0-3e2d89522c0b" (UID: "49d006ce-9d62-4a69-bed0-3e2d89522c0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:20.649318 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.649295    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d006ce-9d62-4a69-bed0-3e2d89522c0b-kube-api-access-65jrs" pod "49d006ce-9d62-4a69-bed0-3e2d89522c0b" (UID: "49d006ce-9d62-4a69-bed0-3e2d89522c0b"). InnerVolumeSpecName "kube-api-access-65jrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:20.748520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.748464    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65jrs\" (UniqueName: \"kubernetes.io/projected/49d006ce-9d62-4a69-bed0-3e2d89522c0b-kube-api-access-65jrs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:20.748520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:20.748486    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d006ce-9d62-4a69-bed0-3e2d89522c0b-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:21.446261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:21.446236    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"
Oct 30 21:54:21.446261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:21.446247    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"
Oct 30 21:54:21.446261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:21.446253    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg" event={"ID":"49d006ce-9d62-4a69-bed0-3e2d89522c0b","Type":"ContainerDied","Data":"f22c6432fcd13bf25ecd420d992c7c2e0fea346aee8e5e7cc5a0e753a2a17f9e"}
Oct 30 21:54:21.446729 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:21.446307    5786 scope.go:122] "RemoveContainer" containerID="85883410d664f62981d4a851a37d99339a1eb93e55eec37291fe836dd6a0fc3c"
Oct 30 21:54:21.606678 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:21.606642    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"]
Oct 30 21:54:21.655836 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:21.655802    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5f31bd-8459f9cc95-s8ktg"]
Oct 30 21:54:21.763654 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:21.763582    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"]
Oct 30 21:54:21.767702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:21.767677    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5f31bd-79b776b9f-nztnp"]
Oct 30 21:54:22.353650 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:54:22.353617    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:54:22.357557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:22.357537    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a2feebb-1715-45b3-b8e5-6c808b94c262" path="/var/lib/kubelet/pods/1a2feebb-1715-45b3-b8e5-6c808b94c262/volumes"
Oct 30 21:54:22.357896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:22.357883    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49d006ce-9d62-4a69-bed0-3e2d89522c0b" path="/var/lib/kubelet/pods/49d006ce-9d62-4a69-bed0-3e2d89522c0b/volumes"
Oct 30 21:54:23.845047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:23.845016    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"]
Oct 30 21:54:23.845538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:23.845198    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb" podUID="4c025712-f515-4f27-a581-1b9e85f3ee04" containerName="maas-api" containerID="cri-o://6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a" gracePeriod=30
Oct 30 21:54:24.095320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.095261    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:54:24.182057 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.182028    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-metrics-certs\") pod \"4c025712-f515-4f27-a581-1b9e85f3ee04\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") "
Oct 30 21:54:24.182255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.182112    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c025712-f515-4f27-a581-1b9e85f3ee04-kube-api-access-w2p6d\" (UniqueName: \"kubernetes.io/projected/4c025712-f515-4f27-a581-1b9e85f3ee04-kube-api-access-w2p6d\") pod \"4c025712-f515-4f27-a581-1b9e85f3ee04\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") "
Oct 30 21:54:24.182255 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.182173    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls\") pod \"4c025712-f515-4f27-a581-1b9e85f3ee04\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") "
Oct 30 21:54:24.182349 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.182257    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c025712-f515-4f27-a581-1b9e85f3ee04-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c025712-f515-4f27-a581-1b9e85f3ee04-openshift-service-ca\") pod \"4c025712-f515-4f27-a581-1b9e85f3ee04\" (UID: \"4c025712-f515-4f27-a581-1b9e85f3ee04\") "
Oct 30 21:54:24.182596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.182574    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c025712-f515-4f27-a581-1b9e85f3ee04-openshift-service-ca" pod "4c025712-f515-4f27-a581-1b9e85f3ee04" (UID: "4c025712-f515-4f27-a581-1b9e85f3ee04"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:24.184438 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.184411    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls" pod "4c025712-f515-4f27-a581-1b9e85f3ee04" (UID: "4c025712-f515-4f27-a581-1b9e85f3ee04"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:24.184438 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.184440    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-metrics-certs" pod "4c025712-f515-4f27-a581-1b9e85f3ee04" (UID: "4c025712-f515-4f27-a581-1b9e85f3ee04"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:24.184584 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.184501    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c025712-f515-4f27-a581-1b9e85f3ee04-kube-api-access-w2p6d" pod "4c025712-f515-4f27-a581-1b9e85f3ee04" (UID: "4c025712-f515-4f27-a581-1b9e85f3ee04"). InnerVolumeSpecName "kube-api-access-w2p6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:24.283583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.283553    5786 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-metrics-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:24.283583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.283579    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2p6d\" (UniqueName: \"kubernetes.io/projected/4c025712-f515-4f27-a581-1b9e85f3ee04-kube-api-access-w2p6d\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:24.283583 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.283590    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c025712-f515-4f27-a581-1b9e85f3ee04-maas-api-tls\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:24.283827 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.283598    5786 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c025712-f515-4f27-a581-1b9e85f3ee04-openshift-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:24.462839 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.462806    5786 generic.go:356] "Generic (PLEG): container finished" podID="4c025712-f515-4f27-a581-1b9e85f3ee04" containerID="6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a" exitCode=0
Oct 30 21:54:24.462986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.462855    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"
Oct 30 21:54:24.462986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.462884    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb" event={"ID":"4c025712-f515-4f27-a581-1b9e85f3ee04","Type":"ContainerDied","Data":"6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a"}
Oct 30 21:54:24.462986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.462937    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb" event={"ID":"4c025712-f515-4f27-a581-1b9e85f3ee04","Type":"ContainerDied","Data":"41819e2b67df90122122c7c787c2768d1e06f13c012c54e52fedda2f0a8adfea"}
Oct 30 21:54:24.462986 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.462959    5786 scope.go:122] "RemoveContainer" containerID="6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a"
Oct 30 21:54:24.472143 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.472128    5786 scope.go:122] "RemoveContainer" containerID="6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a"
Oct 30 21:54:24.472432 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:54:24.472413    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a\": container with ID starting with 6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a not found: ID does not exist" containerID="6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a"
Oct 30 21:54:24.472497 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.472438    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a"} err="failed to get container status \"6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a\": rpc error: code = NotFound desc = could not find container \"6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a\": container with ID starting with 6a66f3dd632fe32f4faba414e18eb27669b1f369e64732b8f68fbc144dced80a not found: ID does not exist"
Oct 30 21:54:24.625079 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.625008    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"]
Oct 30 21:54:24.628402 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:24.628380    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5f31bd-666f5f9749-xqnqb"]
Oct 30 21:54:26.358040 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:26.358000    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c025712-f515-4f27-a581-1b9e85f3ee04" path="/var/lib/kubelet/pods/4c025712-f515-4f27-a581-1b9e85f3ee04/volumes"
Oct 30 21:54:27.410173 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:27.410139    5786 scope.go:122] "RemoveContainer" containerID="1f694bfaf61391f1f8436bc7b02692ec61e963752b31b52ca9d6580a8773bb81"
Oct 30 21:54:27.419103 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:27.419086    5786 scope.go:122] "RemoveContainer" containerID="bb2b3b4c1bdfe14b7c9549dcc136fe559eb4b5a8b341eab6bcca3bac57d3de97"
Oct 30 21:54:27.426876 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:27.426855    5786 scope.go:122] "RemoveContainer" containerID="6b9edf4bdd35a3528a766c68eafc4f7026a27609b54679543848a2f127dfae53"
Oct 30 21:54:27.434556 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:27.434539    5786 scope.go:122] "RemoveContainer" containerID="5dd672e4281b230e16677f37dde8e025568eb3fca0640c4385821e8402cc88fd"
Oct 30 21:54:33.339921 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:33.339842    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"]
Oct 30 21:54:33.340372 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:33.340077    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy" containerID="cri-o://6eea36ee4004a7a717e1fec87c3c102b42d3ddafeba08cc3287aaa7482aadf6c" gracePeriod=30
Oct 30 21:54:34.332856 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.332821    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"]
Oct 30 21:54:34.333258 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333244    5786 cpu_manager.go:414] "containerMap: removing container" podUID="1a2feebb-1715-45b3-b8e5-6c808b94c262" containerName="payload-processing"
Oct 30 21:54:34.333312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333258    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a2feebb-1715-45b3-b8e5-6c808b94c262" containerName="payload-processing"
Oct 30 21:54:34.333312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333270    5786 cpu_manager.go:414] "containerMap: removing container" podUID="977c46a1-044a-4545-8181-58c5ba1b0a18" containerName="payload-processing"
Oct 30 21:54:34.333312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333275    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="977c46a1-044a-4545-8181-58c5ba1b0a18" containerName="payload-processing"
Oct 30 21:54:34.333312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333287    5786 cpu_manager.go:414] "containerMap: removing container" podUID="4c025712-f515-4f27-a581-1b9e85f3ee04" containerName="maas-api"
Oct 30 21:54:34.333312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333292    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c025712-f515-4f27-a581-1b9e85f3ee04" containerName="maas-api"
Oct 30 21:54:34.333312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333304    5786 cpu_manager.go:414] "containerMap: removing container" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy"
Oct 30 21:54:34.333312 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333308    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333320    5786 cpu_manager.go:414] "containerMap: removing container" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333324    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333331    5786 cpu_manager.go:414] "containerMap: removing container" podUID="49d006ce-9d62-4a69-bed0-3e2d89522c0b" containerName="payload-pre-processing"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333337    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49d006ce-9d62-4a69-bed0-3e2d89522c0b" containerName="payload-pre-processing"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333349    5786 cpu_manager.go:414] "containerMap: removing container" podUID="bd16c744-8e51-427c-8ddb-b6fc9508f361" containerName="revoke-keys"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333353    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd16c744-8e51-427c-8ddb-b6fc9508f361" containerName="revoke-keys"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333363    5786 cpu_manager.go:414] "containerMap: removing container" podUID="91bd3b74-19bf-4b72-b892-dc85d1dec133" containerName="payload-pre-processing"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333367    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91bd3b74-19bf-4b72-b892-dc85d1dec133" containerName="payload-pre-processing"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333427    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c025712-f515-4f27-a581-1b9e85f3ee04" containerName="maas-api"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333437    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333443    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="69bebe1c-95dc-4834-b836-bbd28f21f95d" containerName="istio-proxy"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333449    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="49d006ce-9d62-4a69-bed0-3e2d89522c0b" containerName="payload-pre-processing"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333455    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd16c744-8e51-427c-8ddb-b6fc9508f361" containerName="revoke-keys"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333462    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="91bd3b74-19bf-4b72-b892-dc85d1dec133" containerName="payload-pre-processing"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333469    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a2feebb-1715-45b3-b8e5-6c808b94c262" containerName="payload-processing"
Oct 30 21:54:34.333530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.333475    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="977c46a1-044a-4545-8181-58c5ba1b0a18" containerName="payload-processing"
Oct 30 21:54:34.338456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.338437    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.381772 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.381745    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.382167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.382075    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.382167 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.382116    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz8x4\" (UniqueName: \"kubernetes.io/projected/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-kube-api-access-nz8x4\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.394778 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.394752    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"]
Oct 30 21:54:34.441872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.441845    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:54:34.441872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.441849    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:54:34.482669 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.482644    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.482815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.482673    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz8x4\" (UniqueName: \"kubernetes.io/projected/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-kube-api-access-nz8x4\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.482815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.482716    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.485091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.485070    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.491321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.491301    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:54:34.491321 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.491304    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:54:34.493828 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.493809    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.539618 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.539596    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz8x4\" (UniqueName: \"kubernetes.io/projected/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-kube-api-access-nz8x4\") pod \"maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:34.650578 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:34.650527    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:35.024595 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:35.024563    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"]
Oct 30 21:54:35.027842 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:54:35.027816    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af8fe72_987b_47ae_94ce_3c0cd7b5bd0e.slice/crio-2a69a20264cd0fe52936ced6d883e35472bc9aa50bb5863f3f52dac94c3de19b WatchSource:0}: Error finding container 2a69a20264cd0fe52936ced6d883e35472bc9aa50bb5863f3f52dac94c3de19b: Status 404 returned error can't find the container with id 2a69a20264cd0fe52936ced6d883e35472bc9aa50bb5863f3f52dac94c3de19b
Oct 30 21:54:35.528498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:35.528473    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2" event={"ID":"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e","Type":"ContainerStarted","Data":"f3eea40a14904b816d8a6e78c6505f8d21b4039c45bfff0da289b59c00d221c6"}
Oct 30 21:54:35.528834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:35.528520    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2" event={"ID":"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e","Type":"ContainerStarted","Data":"2a69a20264cd0fe52936ced6d883e35472bc9aa50bb5863f3f52dac94c3de19b"}
Oct 30 21:54:35.642794 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:35.642722    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2" podStartSLOduration=1.6427090579999999 podStartE2EDuration="1.642709058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:35.640875368 +0000 UTC m=+2531.511994488" watchObservedRunningTime="2026-10-30 21:54:35.642709058 +0000 UTC m=+2531.513828176"
Oct 30 21:54:36.534538 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:36.534507    5786 generic.go:356] "Generic (PLEG): container finished" podID="4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e" containerID="f3eea40a14904b816d8a6e78c6505f8d21b4039c45bfff0da289b59c00d221c6" exitCode=0
Oct 30 21:54:36.534914 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:36.534562    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2" event={"ID":"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e","Type":"ContainerDied","Data":"f3eea40a14904b816d8a6e78c6505f8d21b4039c45bfff0da289b59c00d221c6"}
Oct 30 21:54:37.670767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.670747    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:37.711740 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.711715    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-xks-ca\") pod \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") "
Oct 30 21:54:37.711840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.711750    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-openshift-ca\") pod \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") "
Oct 30 21:54:37.711840 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.711836    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-kube-api-access-nz8x4\" (UniqueName: \"kubernetes.io/projected/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-kube-api-access-nz8x4\") pod \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\" (UID: \"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e\") "
Oct 30 21:54:37.712124 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.712103    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-openshift-ca" pod "4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e" (UID: "4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:37.713834 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.713813    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-xks-ca" pod "4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e" (UID: "4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:37.713894 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.713855    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-kube-api-access-nz8x4" pod "4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e" (UID: "4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e"). InnerVolumeSpecName "kube-api-access-nz8x4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:37.813146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.813085    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:37.813146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.813109    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:37.813146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:37.813120    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz8x4\" (UniqueName: \"kubernetes.io/projected/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e-kube-api-access-nz8x4\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.547307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.547275    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"
Oct 30 21:54:38.547307 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.547283    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2" event={"ID":"4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e","Type":"ContainerDied","Data":"2a69a20264cd0fe52936ced6d883e35472bc9aa50bb5863f3f52dac94c3de19b"}
Oct 30 21:54:38.547548 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.547333    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a69a20264cd0fe52936ced6d883e35472bc9aa50bb5863f3f52dac94c3de19b"
Oct 30 21:54:38.549818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.549789    5786 generic.go:356] "Generic (PLEG): container finished" podID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerID="6eea36ee4004a7a717e1fec87c3c102b42d3ddafeba08cc3287aaa7482aadf6c" exitCode=0
Oct 30 21:54:38.549935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.549823    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" event={"ID":"0ddcb89a-a9a5-4798-b1c0-24d7da39df87","Type":"ContainerDied","Data":"6eea36ee4004a7a717e1fec87c3c102b42d3ddafeba08cc3287aaa7482aadf6c"}
Oct 30 21:54:38.549935 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.549882    5786 scope.go:122] "RemoveContainer" containerID="53373e78d3d418c17cbf22bade2d68fa8be6a6610ec5ae3488f1bb2b31175fbc"
Oct 30 21:54:38.573456 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.573439    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:54:38.620998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.620950    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-kube-api-access-bqlh9\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-kube-api-access-bqlh9\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.620998 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.620986    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-podinfo\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.621171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621011    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istiod-ca-cert\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.621171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621050    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-socket\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.621171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621093    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-credential-socket\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.621171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621110    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-envoy\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.621171 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621162    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-data\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.621477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621191    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-certs\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.621477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621273    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-token\") pod \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\" (UID: \"0ddcb89a-a9a5-4798-b1c0-24d7da39df87\") "
Oct 30 21:54:38.621477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621404    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-socket" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:38.621477 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621419    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-credential-socket" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:38.621866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621489    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istiod-ca-cert" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:38.621866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621612    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-certs" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:38.621866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621669    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.621866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621691    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.621866 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621705    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.622152 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.621572    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-data" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:38.623164 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.623140    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-podinfo" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:38.623280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.623190    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-kube-api-access-bqlh9" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "kube-api-access-bqlh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:38.623498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.623480    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-token" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:38.623498 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.623486    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-envoy" pod "0ddcb89a-a9a5-4798-b1c0-24d7da39df87" (UID: "0ddcb89a-a9a5-4798-b1c0-24d7da39df87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:38.722576 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.722552    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.722576 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.722574    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.722888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.722584    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.722888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.722592    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqlh9\" (UniqueName: \"kubernetes.io/projected/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-kube-api-access-bqlh9\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.722888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.722601    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.722888 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.722608    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcb89a-a9a5-4798-b1c0-24d7da39df87-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.790139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:38.790113    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"]
Oct 30 21:54:39.557031 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:39.556997    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"
Oct 30 21:54:39.557231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:39.556997    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc" event={"ID":"0ddcb89a-a9a5-4798-b1c0-24d7da39df87","Type":"ContainerDied","Data":"0a60d3118b1a0858ffb9d2b712021f0afc13487adec0f7dd5332ac32c83b4565"}
Oct 30 21:54:39.557231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:39.557132    5786 scope.go:122] "RemoveContainer" containerID="6eea36ee4004a7a717e1fec87c3c102b42d3ddafeba08cc3287aaa7482aadf6c"
Oct 30 21:54:39.718446 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:39.718417    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"]
Oct 30 21:54:39.767601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:39.767577    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5f31bd-openshift-default-75854454df-mkgnc"]
Oct 30 21:54:40.358251 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:40.358201    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" path="/var/lib/kubelet/pods/0ddcb89a-a9a5-4798-b1c0-24d7da39df87/volumes"
Oct 30 21:54:45.266751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.266714    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"]
Oct 30 21:54:45.267348 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267329    5786 cpu_manager.go:414] "containerMap: removing container" podUID="4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e" containerName="revoke-keys"
Oct 30 21:54:45.267444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267350    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e" containerName="revoke-keys"
Oct 30 21:54:45.267444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267368    5786 cpu_manager.go:414] "containerMap: removing container" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy"
Oct 30 21:54:45.267444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267375    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy"
Oct 30 21:54:45.267444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267412    5786 cpu_manager.go:414] "containerMap: removing container" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy"
Oct 30 21:54:45.267444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267422    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy"
Oct 30 21:54:45.267703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267519    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e" containerName="revoke-keys"
Oct 30 21:54:45.267703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267531    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy"
Oct 30 21:54:45.267703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.267545    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ddcb89a-a9a5-4798-b1c0-24d7da39df87" containerName="istio-proxy"
Oct 30 21:54:45.270688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.270667    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.324551 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.324505    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:54:45.374081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.373913    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:54:45.374081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.373973    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:54:45.374081 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.374053    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:54:45.374396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.374335    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.374467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.374447    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.374516 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.374484    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6cn6\" (UniqueName: \"kubernetes.io/projected/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-kube-api-access-m6cn6\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.375932 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.375909    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"]
Oct 30 21:54:45.475467 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.475443    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.475593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.475518    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.475593 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.475550    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cn6\" (UniqueName: \"kubernetes.io/projected/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-kube-api-access-m6cn6\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.476153 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.476135    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.477860 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.477838    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.531554 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.531495    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6cn6\" (UniqueName: \"kubernetes.io/projected/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-kube-api-access-m6cn6\") pod \"maas-api-revoke-sub-5e62e732-fml8m\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.582671 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.582648    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:45.627033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.626991    5786 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal" podUID="d2a11b2240e0ab276a762b4e71b7c25e" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:54:45.914179 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.914154    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"]
Oct 30 21:54:45.952093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:45.952067    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"]
Oct 30 21:54:45.955382 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:54:45.955351    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1524c15_b9d0_460d_86ea_a7e5c1f566bd.slice/crio-5c5eac86e0ca9fdd70af3d41f57c25e6d171361a19f7898627b5eb9acf357483 WatchSource:0}: Error finding container 5c5eac86e0ca9fdd70af3d41f57c25e6d171361a19f7898627b5eb9acf357483: Status 404 returned error can't find the container with id 5c5eac86e0ca9fdd70af3d41f57c25e6d171361a19f7898627b5eb9acf357483
Oct 30 21:54:46.592819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:46.592730    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m" event={"ID":"d1524c15-b9d0-460d-86ea-a7e5c1f566bd","Type":"ContainerStarted","Data":"2d0403e4cdffbff023c82d6b0fe758e7c03331c4e1b6b0db9c59b4cc301ede3f"}
Oct 30 21:54:46.592819 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:46.592781    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m" event={"ID":"d1524c15-b9d0-460d-86ea-a7e5c1f566bd","Type":"ContainerStarted","Data":"5c5eac86e0ca9fdd70af3d41f57c25e6d171361a19f7898627b5eb9acf357483"}
Oct 30 21:54:46.706443 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:46.706385    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m" podStartSLOduration=1.70637111 podStartE2EDuration="1.70637111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:46.705415028 +0000 UTC m=+2542.576534231" watchObservedRunningTime="2026-10-30 21:54:46.70637111 +0000 UTC m=+2542.577490242"
Oct 30 21:54:47.599373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:47.599343    5786 generic.go:356] "Generic (PLEG): container finished" podID="d1524c15-b9d0-460d-86ea-a7e5c1f566bd" containerID="2d0403e4cdffbff023c82d6b0fe758e7c03331c4e1b6b0db9c59b4cc301ede3f" exitCode=0
Oct 30 21:54:47.599786 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:47.599466    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m" event={"ID":"d1524c15-b9d0-460d-86ea-a7e5c1f566bd","Type":"ContainerDied","Data":"2d0403e4cdffbff023c82d6b0fe758e7c03331c4e1b6b0db9c59b4cc301ede3f"}
Oct 30 21:54:48.048661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.048632    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"]
Oct 30 21:54:48.051005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.050980    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b82c593-98bld"]
Oct 30 21:54:48.358253 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.358162    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="568d4be3-420b-4b37-af8b-51474561b8dc" path="/var/lib/kubelet/pods/568d4be3-420b-4b37-af8b-51474561b8dc/volumes"
Oct 30 21:54:48.741002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.740981    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:48.808496 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.808464    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-kube-api-access-m6cn6\" (UniqueName: \"kubernetes.io/projected/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-kube-api-access-m6cn6\") pod \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") "
Oct 30 21:54:48.808634 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.808616    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-openshift-ca\") pod \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") "
Oct 30 21:54:48.808682 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.808661    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-xks-ca\") pod \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\" (UID: \"d1524c15-b9d0-460d-86ea-a7e5c1f566bd\") "
Oct 30 21:54:48.808953 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.808930    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-openshift-ca" pod "d1524c15-b9d0-460d-86ea-a7e5c1f566bd" (UID: "d1524c15-b9d0-460d-86ea-a7e5c1f566bd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:48.809099 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.809080    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.810653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.810635    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-xks-ca" pod "d1524c15-b9d0-460d-86ea-a7e5c1f566bd" (UID: "d1524c15-b9d0-460d-86ea-a7e5c1f566bd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:48.810717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.810651    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-kube-api-access-m6cn6" pod "d1524c15-b9d0-460d-86ea-a7e5c1f566bd" (UID: "d1524c15-b9d0-460d-86ea-a7e5c1f566bd"). InnerVolumeSpecName "kube-api-access-m6cn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:48.909826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.909761    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6cn6\" (UniqueName: \"kubernetes.io/projected/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-kube-api-access-m6cn6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.909826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:48.909786    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1524c15-b9d0-460d-86ea-a7e5c1f566bd-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:49.610393 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:49.610359    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"
Oct 30 21:54:49.610575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:49.610364    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m" event={"ID":"d1524c15-b9d0-460d-86ea-a7e5c1f566bd","Type":"ContainerDied","Data":"5c5eac86e0ca9fdd70af3d41f57c25e6d171361a19f7898627b5eb9acf357483"}
Oct 30 21:54:49.610575 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:49.610467    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c5eac86e0ca9fdd70af3d41f57c25e6d171361a19f7898627b5eb9acf357483"
Oct 30 21:54:50.914950 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:50.914901    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j" podUID="22504f51-bb1e-499a-b14a-cd422f024088" containerName="payload-pre-processing" containerID="cri-o://6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88" gracePeriod=25
Oct 30 21:54:52.352896 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:54:52.352865    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:54:53.809601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:53.809573    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main" containerID="cri-o://1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c" gracePeriod=45
Oct 30 21:54:54.058159 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.058135    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:54:54.157923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.157900    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-dshm\") pod \"bae5037c-823b-4c3d-9ec1-76ce3d142741\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") "
Oct 30 21:54:54.158059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.157936    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bae5037c-823b-4c3d-9ec1-76ce3d142741-kube-api-access-tjmwt\" (UniqueName: \"kubernetes.io/projected/bae5037c-823b-4c3d-9ec1-76ce3d142741-kube-api-access-tjmwt\") pod \"bae5037c-823b-4c3d-9ec1-76ce3d142741\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") "
Oct 30 21:54:54.158059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.157977    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-home\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-home\") pod \"bae5037c-823b-4c3d-9ec1-76ce3d142741\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") "
Oct 30 21:54:54.158059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.158034    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-tmp-dir\") pod \"bae5037c-823b-4c3d-9ec1-76ce3d142741\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") "
Oct 30 21:54:54.158275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.158099    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bae5037c-823b-4c3d-9ec1-76ce3d142741-tls-certs\" (UniqueName: \"kubernetes.io/secret/bae5037c-823b-4c3d-9ec1-76ce3d142741-tls-certs\") pod \"bae5037c-823b-4c3d-9ec1-76ce3d142741\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") "
Oct 30 21:54:54.158275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.158119    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-model-cache\") pod \"bae5037c-823b-4c3d-9ec1-76ce3d142741\" (UID: \"bae5037c-823b-4c3d-9ec1-76ce3d142741\") "
Oct 30 21:54:54.158275 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.158198    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-home" pod "bae5037c-823b-4c3d-9ec1-76ce3d142741" (UID: "bae5037c-823b-4c3d-9ec1-76ce3d142741"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:54.158464 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.158289    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-tmp-dir" pod "bae5037c-823b-4c3d-9ec1-76ce3d142741" (UID: "bae5037c-823b-4c3d-9ec1-76ce3d142741"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:54.158726 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.158699    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-model-cache" pod "bae5037c-823b-4c3d-9ec1-76ce3d142741" (UID: "bae5037c-823b-4c3d-9ec1-76ce3d142741"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:54.159047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.158958    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:54.159047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.158999    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:54.159047 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.159016    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:54.160493 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.160470    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-dshm" pod "bae5037c-823b-4c3d-9ec1-76ce3d142741" (UID: "bae5037c-823b-4c3d-9ec1-76ce3d142741"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:54.160645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.160609    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae5037c-823b-4c3d-9ec1-76ce3d142741-kube-api-access-tjmwt" pod "bae5037c-823b-4c3d-9ec1-76ce3d142741" (UID: "bae5037c-823b-4c3d-9ec1-76ce3d142741"). InnerVolumeSpecName "kube-api-access-tjmwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:54.160893 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.160873    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae5037c-823b-4c3d-9ec1-76ce3d142741-tls-certs" pod "bae5037c-823b-4c3d-9ec1-76ce3d142741" (UID: "bae5037c-823b-4c3d-9ec1-76ce3d142741"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:54.259970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.259937    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bae5037c-823b-4c3d-9ec1-76ce3d142741-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:54.259970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.259965    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bae5037c-823b-4c3d-9ec1-76ce3d142741-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:54.259970 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.259976    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjmwt\" (UniqueName: \"kubernetes.io/projected/bae5037c-823b-4c3d-9ec1-76ce3d142741-kube-api-access-tjmwt\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:54:54.636661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.636586    5786 generic.go:356] "Generic (PLEG): container finished" podID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerID="1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c" exitCode=0
Oct 30 21:54:54.636661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.636636    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"
Oct 30 21:54:54.636661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.636642    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" event={"ID":"bae5037c-823b-4c3d-9ec1-76ce3d142741","Type":"ContainerDied","Data":"1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c"}
Oct 30 21:54:54.636889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.636675    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r" event={"ID":"bae5037c-823b-4c3d-9ec1-76ce3d142741","Type":"ContainerDied","Data":"dceefa7a2a0dd0357e8f39b58bb383d449f70f0841babc2c9bd05288f37765cb"}
Oct 30 21:54:54.636889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.636691    5786 scope.go:122] "RemoveContainer" containerID="1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c"
Oct 30 21:54:54.645185 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.645168    5786 scope.go:122] "RemoveContainer" containerID="1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c"
Oct 30 21:54:54.645449 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:54:54.645431    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c\": container with ID starting with 1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c not found: ID does not exist" containerID="1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c"
Oct 30 21:54:54.645513 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.645458    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c"} err="failed to get container status \"1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c\": rpc error: code = NotFound desc = could not find container \"1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c\": container with ID starting with 1f80d2045d296e6040ec5a1c67912788d8eefdb4b0ec7f2f26e2a13271c7ce8c not found: ID does not exist"
Oct 30 21:54:54.799318 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.799293    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"]
Oct 30 21:54:54.848599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:54.848576    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-8ce31a/simulator-w1-8ce31a-kserve-84c4974b9-xbh6r"]
Oct 30 21:54:56.356976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:54:56.356944    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" path="/var/lib/kubelet/pods/bae5037c-823b-4c3d-9ec1-76ce3d142741/volumes"
Oct 30 21:55:00.701313 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:00.701279    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j" podUID="22504f51-bb1e-499a-b14a-cd422f024088" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:01.711421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:01.711388    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"]
Oct 30 21:55:01.760399 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:01.760368    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-8ce31a-cmkk2"]
Oct 30 21:55:02.357375 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:02.357350    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e" path="/var/lib/kubelet/pods/4af8fe72-987b-47ae-94ce-3c0cd7b5bd0e/volumes"
Oct 30 21:55:10.700857 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:10.700818    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j" podUID="22504f51-bb1e-499a-b14a-cd422f024088" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:11.162717 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.162695    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:55:11.305707 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.305644    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22504f51-bb1e-499a-b14a-cd422f024088-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22504f51-bb1e-499a-b14a-cd422f024088-plugins-config-volume\") pod \"22504f51-bb1e-499a-b14a-cd422f024088\" (UID: \"22504f51-bb1e-499a-b14a-cd422f024088\") "
Oct 30 21:55:11.305707 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.305705    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22504f51-bb1e-499a-b14a-cd422f024088-kube-api-access-cts7m\" (UniqueName: \"kubernetes.io/projected/22504f51-bb1e-499a-b14a-cd422f024088-kube-api-access-cts7m\") pod \"22504f51-bb1e-499a-b14a-cd422f024088\" (UID: \"22504f51-bb1e-499a-b14a-cd422f024088\") "
Oct 30 21:55:11.306054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.306030    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22504f51-bb1e-499a-b14a-cd422f024088-plugins-config-volume" pod "22504f51-bb1e-499a-b14a-cd422f024088" (UID: "22504f51-bb1e-499a-b14a-cd422f024088"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:11.307753 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.307732    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22504f51-bb1e-499a-b14a-cd422f024088-kube-api-access-cts7m" pod "22504f51-bb1e-499a-b14a-cd422f024088" (UID: "22504f51-bb1e-499a-b14a-cd422f024088"). InnerVolumeSpecName "kube-api-access-cts7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:11.406491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.406467    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cts7m\" (UniqueName: \"kubernetes.io/projected/22504f51-bb1e-499a-b14a-cd422f024088-kube-api-access-cts7m\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:11.406491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.406490    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22504f51-bb1e-499a-b14a-cd422f024088-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:11.717478 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.717450    5786 generic.go:356] "Generic (PLEG): container finished" podID="22504f51-bb1e-499a-b14a-cd422f024088" containerID="6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88" exitCode=0
Oct 30 21:55:11.717887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.717502    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"
Oct 30 21:55:11.717887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.717528    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j" event={"ID":"22504f51-bb1e-499a-b14a-cd422f024088","Type":"ContainerDied","Data":"6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88"}
Oct 30 21:55:11.717887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.717575    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j" event={"ID":"22504f51-bb1e-499a-b14a-cd422f024088","Type":"ContainerDied","Data":"92f721b42fad273167c792f8f43dc1f54a0168db6118630fa295674c5f210136"}
Oct 30 21:55:11.717887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.717597    5786 scope.go:122] "RemoveContainer" containerID="6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88"
Oct 30 21:55:11.726520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.726507    5786 scope.go:122] "RemoveContainer" containerID="6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88"
Oct 30 21:55:11.726713 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:55:11.726699    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88\": container with ID starting with 6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88 not found: ID does not exist" containerID="6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88"
Oct 30 21:55:11.726773 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.726719    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88"} err="failed to get container status \"6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88\": rpc error: code = NotFound desc = could not find container \"6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88\": container with ID starting with 6e7e13bc2dc1d2377863d19c4afb3850d3b333914a93f007221c228ecc10bf88 not found: ID does not exist"
Oct 30 21:55:11.879300 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.879278    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"]
Oct 30 21:55:11.928621 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:11.928601    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8ce31a-9785d8c9-bkg7j"]
Oct 30 21:55:12.357270 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:12.357248    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22504f51-bb1e-499a-b14a-cd422f024088" path="/var/lib/kubelet/pods/22504f51-bb1e-499a-b14a-cd422f024088/volumes"
Oct 30 21:55:22.352941 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:55:22.352905    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:55:27.491886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:27.491852    5786 scope.go:122] "RemoveContainer" containerID="63355943e1b0de3ca3ed88c615d8fb8fa2472d85d4a649330f8aced523915078"
Oct 30 21:55:27.499941 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:27.499922    5786 scope.go:122] "RemoveContainer" containerID="3564d67ca0aa65681705c7401ad57fca0278684e21b7f40837c660de79a4e401"
Oct 30 21:55:30.049560 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:30.049532    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"]
Oct 30 21:55:30.098639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:30.098615    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a01485f4-smdff"]
Oct 30 21:55:30.357777 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:30.357707    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bc433cb-99ff-4a5e-a250-00dfa44ed156" path="/var/lib/kubelet/pods/7bc433cb-99ff-4a5e-a250-00dfa44ed156/volumes"
Oct 30 21:55:50.037999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.037964    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"]
Oct 30 21:55:50.038488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038386    5786 cpu_manager.go:414] "containerMap: removing container" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main"
Oct 30 21:55:50.038488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038399    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main"
Oct 30 21:55:50.038488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038429    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d1524c15-b9d0-460d-86ea-a7e5c1f566bd" containerName="revoke-keys"
Oct 30 21:55:50.038488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038435    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1524c15-b9d0-460d-86ea-a7e5c1f566bd" containerName="revoke-keys"
Oct 30 21:55:50.038488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038452    5786 cpu_manager.go:414] "containerMap: removing container" podUID="22504f51-bb1e-499a-b14a-cd422f024088" containerName="payload-pre-processing"
Oct 30 21:55:50.038488 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038457    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22504f51-bb1e-499a-b14a-cd422f024088" containerName="payload-pre-processing"
Oct 30 21:55:50.038687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038511    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="22504f51-bb1e-499a-b14a-cd422f024088" containerName="payload-pre-processing"
Oct 30 21:55:50.038687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038521    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="bae5037c-823b-4c3d-9ec1-76ce3d142741" containerName="main"
Oct 30 21:55:50.038687 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.038531    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1524c15-b9d0-460d-86ea-a7e5c1f566bd" containerName="revoke-keys"
Oct 30 21:55:50.042764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.042748    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.097748 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.097721    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:55:50.131254 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.131199    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5rhk\" (UniqueName: \"kubernetes.io/projected/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-kube-api-access-k5rhk\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.131391 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.131301    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.131454 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.131440    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.143977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.143955    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:55:50.143977 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.143970    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:55:50.144147 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.143986    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:55:50.146714 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.146676    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"]
Oct 30 21:55:50.232872 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.232839    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.233020 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.232894    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5rhk\" (UniqueName: \"kubernetes.io/projected/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-kube-api-access-k5rhk\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.233020 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.232930    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.233499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.233469    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.235236 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.235217    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.289469 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.289412    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5rhk\" (UniqueName: \"kubernetes.io/projected/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-kube-api-access-k5rhk\") pod \"maas-api-revoke-sub-85237e45-fsc5x\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.357534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.357507    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:50.730459 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.730430    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"]
Oct 30 21:55:50.733911 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:55:50.733882    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda668a3e8_c9b1_455e_bf42_973b1c7f3cc3.slice/crio-3caff7f00f05d0a77debcf68d9f187cd2502e0444e8f497546013bbe367fae91 WatchSource:0}: Error finding container 3caff7f00f05d0a77debcf68d9f187cd2502e0444e8f497546013bbe367fae91: Status 404 returned error can't find the container with id 3caff7f00f05d0a77debcf68d9f187cd2502e0444e8f497546013bbe367fae91
Oct 30 21:55:50.910987 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:50.910954    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x" event={"ID":"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3","Type":"ContainerStarted","Data":"3caff7f00f05d0a77debcf68d9f187cd2502e0444e8f497546013bbe367fae91"}
Oct 30 21:55:51.917780 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:51.917748    5786 generic.go:356] "Generic (PLEG): container finished" podID="a668a3e8-c9b1-455e-bf42-973b1c7f3cc3" containerID="37a6ff76a8cc9f254e9053de5f5bee19a049063fd9b38304b925cbb37e95caaf" exitCode=0
Oct 30 21:55:51.918154 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:51.917831    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x" event={"ID":"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3","Type":"ContainerDied","Data":"37a6ff76a8cc9f254e9053de5f5bee19a049063fd9b38304b925cbb37e95caaf"}
Oct 30 21:55:52.355445 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:55:52.355357    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:55:53.051972 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.051944    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:53.155598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.155569    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-xks-ca\") pod \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") "
Oct 30 21:55:53.155756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.155615    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-openshift-ca\") pod \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") "
Oct 30 21:55:53.155756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.155650    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-kube-api-access-k5rhk\" (UniqueName: \"kubernetes.io/projected/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-kube-api-access-k5rhk\") pod \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\" (UID: \"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3\") "
Oct 30 21:55:53.156041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.156014    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-openshift-ca" pod "a668a3e8-c9b1-455e-bf42-973b1c7f3cc3" (UID: "a668a3e8-c9b1-455e-bf42-973b1c7f3cc3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:53.157645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.157626    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-xks-ca" pod "a668a3e8-c9b1-455e-bf42-973b1c7f3cc3" (UID: "a668a3e8-c9b1-455e-bf42-973b1c7f3cc3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:53.157965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.157945    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-kube-api-access-k5rhk" pod "a668a3e8-c9b1-455e-bf42-973b1c7f3cc3" (UID: "a668a3e8-c9b1-455e-bf42-973b1c7f3cc3"). InnerVolumeSpecName "kube-api-access-k5rhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:53.257449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.257375    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:53.257449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.257405    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:53.257449 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.257420    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5rhk\" (UniqueName: \"kubernetes.io/projected/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3-kube-api-access-k5rhk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:53.928665 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.928636    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"
Oct 30 21:55:53.928832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.928674    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x" event={"ID":"a668a3e8-c9b1-455e-bf42-973b1c7f3cc3","Type":"ContainerDied","Data":"3caff7f00f05d0a77debcf68d9f187cd2502e0444e8f497546013bbe367fae91"}
Oct 30 21:55:53.928832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:55:53.928718    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3caff7f00f05d0a77debcf68d9f187cd2502e0444e8f497546013bbe367fae91"
Oct 30 21:56:04.272021 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.271944    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"]
Oct 30 21:56:04.272444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.272389    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a668a3e8-c9b1-455e-bf42-973b1c7f3cc3" containerName="revoke-keys"
Oct 30 21:56:04.272444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.272401    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a668a3e8-c9b1-455e-bf42-973b1c7f3cc3" containerName="revoke-keys"
Oct 30 21:56:04.272520 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.272474    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a668a3e8-c9b1-455e-bf42-973b1c7f3cc3" containerName="revoke-keys"
Oct 30 21:56:04.275615 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.275596    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358004    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358036    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358064    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358118    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358383 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358271    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad04b60-4d70-44de-badb-f602b63d1556-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358383 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358321    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tftxl\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-kube-api-access-tftxl\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358416    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad04b60-4d70-44de-badb-f602b63d1556-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358455    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.358494 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.358485    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.379043 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.379022    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"]
Oct 30 21:56:04.459782 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.459759    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.459939 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.459787    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.459939 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.459817    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.459943    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460018    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460059    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460120    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad04b60-4d70-44de-badb-f602b63d1556-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460178 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460162    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tftxl\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-kube-api-access-tftxl\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460182    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460245    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460273    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad04b60-4d70-44de-badb-f602b63d1556-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460419    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460499 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460469    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.460763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.460741    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad04b60-4d70-44de-badb-f602b63d1556-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.462226 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.462189    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.462673 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.462655    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad04b60-4d70-44de-badb-f602b63d1556-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.516645 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.516611    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.516953 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.516934    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tftxl\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-kube-api-access-tftxl\") pod \"maas-default-gateway-openshift-default-6cc458d887-xtfjt\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:04.588500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:04.588445    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:05.017433 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:05.017405    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"]
Oct 30 21:56:05.021023 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:56:05.020998    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad04b60_4d70_44de_badb_f602b63d1556.slice/crio-8f0ff34af0727484f4f17ad80f3fc2c33cd28d32b797b11de88ed023b91787d8 WatchSource:0}: Error finding container 8f0ff34af0727484f4f17ad80f3fc2c33cd28d32b797b11de88ed023b91787d8: Status 404 returned error can't find the container with id 8f0ff34af0727484f4f17ad80f3fc2c33cd28d32b797b11de88ed023b91787d8
Oct 30 21:56:05.023030 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:05.023002    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:56:05.023087 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:05.023063    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:56:05.023130 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:05.023088    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:56:05.987468 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:05.987422    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" event={"ID":"9ad04b60-4d70-44de-badb-f602b63d1556","Type":"ContainerStarted","Data":"ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1"}
Oct 30 21:56:05.987945 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:05.987530    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" event={"ID":"9ad04b60-4d70-44de-badb-f602b63d1556","Type":"ContainerStarted","Data":"8f0ff34af0727484f4f17ad80f3fc2c33cd28d32b797b11de88ed023b91787d8"}
Oct 30 21:56:06.212181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:06.212131    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podStartSLOduration=2.2121188370000002 podStartE2EDuration="2.212118837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:06.162661965 +0000 UTC m=+2622.033781096" watchObservedRunningTime="2026-10-30 21:56:06.212118837 +0000 UTC m=+2622.083237955"
Oct 30 21:56:06.589582 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:06.589543    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:06.591154 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:06.591128    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:56:06.591342 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:06.591178    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused"
Oct 30 21:56:07.589747 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:07.589713    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:56:07.590181 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:07.589765    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused"
Oct 30 21:56:08.589751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:08.589708    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:56:08.590295 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:08.589785    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused"
Oct 30 21:56:09.588833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:09.588797    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:56:09.589029 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:09.588861    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused"
Oct 30 21:56:10.589567 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:10.589531    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:56:10.589981 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:10.589585    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused"
Oct 30 21:56:11.589768 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:11.589732    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:56:11.590169 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:11.589795    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": dial tcp 10.134.0.218:15021: connect: connection refused"
Oct 30 21:56:12.975910 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:12.975878    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:12.976440 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:12.976229    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:12.977033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:12.977018    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 21:56:22.353244 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:56:22.353190    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:56:27.533059 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:27.533028    5786 scope.go:122] "RemoveContainer" containerID="21815cd86d3ddee164702e5c699861d78a2777c1debd9c46fb2135ae7daf44d3"
Oct 30 21:56:27.541536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:27.541501    5786 scope.go:122] "RemoveContainer" containerID="b3a25dbd8aebbda97fddb50bd5a4a49d7dbe055dec98d794ec4048edcf125c5c"
Oct 30 21:56:27.549305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:27.549287    5786 scope.go:122] "RemoveContainer" containerID="6d68bac060ba6a232581f4765262382dcaa3dee8e3e282a534b5c5f6ba58a2c9"
Oct 30 21:56:27.556795 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:56:27.556779    5786 scope.go:122] "RemoveContainer" containerID="8fdf6141854ee31d7506ddc79e88c767d8eb4977b0e9be20b0e6903a7475fd2b"
Oct 30 21:56:52.354740 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:56:52.354697    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:57:00.171792 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.171760    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"]
Oct 30 21:57:00.175525 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.175504    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.230296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.230269    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:57:00.276679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.276650    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:57:00.276679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.276653    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:57:00.276679 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.276651    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:57:00.278376 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.278357    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"]
Oct 30 21:57:00.349536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.349511    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hq67\" (UniqueName: \"kubernetes.io/projected/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-kube-api-access-5hq67\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.349664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.349562    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.349664 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.349657    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.450417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.450347    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hq67\" (UniqueName: \"kubernetes.io/projected/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-kube-api-access-5hq67\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.450417 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.450411    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.450606 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.450454    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.450999 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.450981    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.452898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.452881    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.507200 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.507172    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hq67\" (UniqueName: \"kubernetes.io/projected/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-kube-api-access-5hq67\") pod \"maas-api-revoke-sub-c1e271a9-rbbg8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:00.788129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:00.788082    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:01.157186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:01.157112    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"]
Oct 30 21:57:01.160100 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:57:01.160062    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod016aeeb8_a2f8_4c34_9956_f2dbc02b76e8.slice/crio-2ee349e9b7e45dc5fd07920cecec7c8f24a940a4e0fd2b86098a6f95f0649d78 WatchSource:0}: Error finding container 2ee349e9b7e45dc5fd07920cecec7c8f24a940a4e0fd2b86098a6f95f0649d78: Status 404 returned error can't find the container with id 2ee349e9b7e45dc5fd07920cecec7c8f24a940a4e0fd2b86098a6f95f0649d78
Oct 30 21:57:01.256858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:01.256829    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8" event={"ID":"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8","Type":"ContainerStarted","Data":"2ee349e9b7e45dc5fd07920cecec7c8f24a940a4e0fd2b86098a6f95f0649d78"}
Oct 30 21:57:02.263299 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:02.263273    5786 generic.go:356] "Generic (PLEG): container finished" podID="016aeeb8-a2f8-4c34-9956-f2dbc02b76e8" containerID="dcd9133f968a73b3708645e4c850e79df1b553484a07f1abdb0c54576efe55f2" exitCode=0
Oct 30 21:57:02.263661 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:02.263355    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8" event={"ID":"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8","Type":"ContainerDied","Data":"dcd9133f968a73b3708645e4c850e79df1b553484a07f1abdb0c54576efe55f2"}
Oct 30 21:57:03.402669 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.402646    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:03.471280 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.471257    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-openshift-ca\") pod \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") "
Oct 30 21:57:03.471373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.471350    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-xks-ca\") pod \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") "
Oct 30 21:57:03.471455 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.471439    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-kube-api-access-5hq67\" (UniqueName: \"kubernetes.io/projected/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-kube-api-access-5hq67\") pod \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\" (UID: \"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8\") "
Oct 30 21:57:03.471632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.471613    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-openshift-ca" pod "016aeeb8-a2f8-4c34-9956-f2dbc02b76e8" (UID: "016aeeb8-a2f8-4c34-9956-f2dbc02b76e8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:03.471742 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.471728    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:03.473350 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.473329    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-xks-ca" pod "016aeeb8-a2f8-4c34-9956-f2dbc02b76e8" (UID: "016aeeb8-a2f8-4c34-9956-f2dbc02b76e8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:03.473426 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.473391    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-kube-api-access-5hq67" pod "016aeeb8-a2f8-4c34-9956-f2dbc02b76e8" (UID: "016aeeb8-a2f8-4c34-9956-f2dbc02b76e8"). InnerVolumeSpecName "kube-api-access-5hq67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:03.572660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.572603    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:03.572660 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:03.572627    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hq67\" (UniqueName: \"kubernetes.io/projected/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8-kube-api-access-5hq67\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:04.273423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:04.273397    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"
Oct 30 21:57:04.273588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:04.273422    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8" event={"ID":"016aeeb8-a2f8-4c34-9956-f2dbc02b76e8","Type":"ContainerDied","Data":"2ee349e9b7e45dc5fd07920cecec7c8f24a940a4e0fd2b86098a6f95f0649d78"}
Oct 30 21:57:04.273588 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:04.273461    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ee349e9b7e45dc5fd07920cecec7c8f24a940a4e0fd2b86098a6f95f0649d78"
Oct 30 21:57:19.857536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.857501    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"]
Oct 30 21:57:19.857952 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.857938    5786 cpu_manager.go:414] "containerMap: removing container" podUID="016aeeb8-a2f8-4c34-9956-f2dbc02b76e8" containerName="revoke-keys"
Oct 30 21:57:19.857996 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.857954    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="016aeeb8-a2f8-4c34-9956-f2dbc02b76e8" containerName="revoke-keys"
Oct 30 21:57:19.858064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.858054    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="016aeeb8-a2f8-4c34-9956-f2dbc02b76e8" containerName="revoke-keys"
Oct 30 21:57:19.861196 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.861177    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.906738 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.906707    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"]
Oct 30 21:57:19.910115 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.910099    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:19.915574 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915551    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.915712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915610    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.915712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915656    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.915712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915674    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.915712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915701    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.915938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915715    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xldsg\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-kube-api-access-xldsg\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.915938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915741    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.915938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915765    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.915938 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.915797    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:19.964874 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:19.964843    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"]
Oct 30 21:57:20.017881 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.014305    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"]
Oct 30 21:57:20.018864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.018712    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq44p\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-kube-api-access-bq44p\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.019120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.019103    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.019681 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.019660    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.020442 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.020421    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.020696 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.020676    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.021329 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.021312    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldsg\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-kube-api-access-xldsg\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.021757 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.021739    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.021982 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.021966    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.022170 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.021248    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.022368 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.022339    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.022565 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.022548    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.022731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.022713    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.022885 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.022870    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.023305 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.023284    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.023423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.023328    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.023423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.023370    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.023423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.023416    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.023611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.023473    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.023611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.023509    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ae86a1-5c75-45ff-b549-32aa463d30a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.024066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.023989    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.024480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.024439    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.024825 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.024803    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.025492 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.025472    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.025837 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.025791    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.027763 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.027740    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.075742 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.075704    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.077219 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.077181    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xldsg\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-kube-api-access-xldsg\") pod \"maas-default-gateway-openshift-default-6cc458d887-6vwl6\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.125054 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.124980    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ae86a1-5c75-45ff-b549-32aa463d30a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.125302 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.125281    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq44p\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-kube-api-access-bq44p\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.125563 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.125544    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.125761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.125742    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.126282 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.126253    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.126731 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.126708    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.127163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.127143    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.127649 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.127631    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.127833 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.127815    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.127984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.126088    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ae86a1-5c75-45ff-b549-32aa463d30a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.128091 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.126167    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.130104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.127578    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.130104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.128674    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.130104 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.129024    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.130363 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.130155    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.130557 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.130530    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.174480 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.174456    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:20.184131 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.184096    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.184631 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.184609    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq44p\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-kube-api-access-bq44p\") pod \"maas-default-gateway-openshift-default-6cc458d887-8b756\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.223756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.223527    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:20.604533 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.604509    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"]
Oct 30 21:57:20.606904 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:57:20.606879    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065139b3_6b4a_450b_a2ec_d8a8cf01caf3.slice/crio-e4b2812cc9fd98de24bfdc97b40b27615fc2aff275c385464c15e6027827b21c WatchSource:0}: Error finding container e4b2812cc9fd98de24bfdc97b40b27615fc2aff275c385464c15e6027827b21c: Status 404 returned error can't find the container with id e4b2812cc9fd98de24bfdc97b40b27615fc2aff275c385464c15e6027827b21c
Oct 30 21:57:20.608962 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.608932    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:57:20.609041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.608993    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:57:20.609041 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.609020    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:57:20.657476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.657452    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"]
Oct 30 21:57:20.659630 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:57:20.659609    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ae86a1_5c75_45ff_b549_32aa463d30a7.slice/crio-4f047f7e96974f7d4ca9335ccd17ee279e8ff8996266c20a18f9de2d348fa01e WatchSource:0}: Error finding container 4f047f7e96974f7d4ca9335ccd17ee279e8ff8996266c20a18f9de2d348fa01e: Status 404 returned error can't find the container with id 4f047f7e96974f7d4ca9335ccd17ee279e8ff8996266c20a18f9de2d348fa01e
Oct 30 21:57:20.661798 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.661772    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:57:20.661900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.661842    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:57:20.661900 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:20.661884    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 21:57:21.387849 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:21.387814    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" event={"ID":"065139b3-6b4a-450b-a2ec-d8a8cf01caf3","Type":"ContainerStarted","Data":"f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d"}
Oct 30 21:57:21.388481 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:21.388456    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" event={"ID":"065139b3-6b4a-450b-a2ec-d8a8cf01caf3","Type":"ContainerStarted","Data":"e4b2812cc9fd98de24bfdc97b40b27615fc2aff275c385464c15e6027827b21c"}
Oct 30 21:57:21.399783 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:21.399749    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" event={"ID":"15ae86a1-5c75-45ff-b549-32aa463d30a7","Type":"ContainerStarted","Data":"0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0"}
Oct 30 21:57:21.400060 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:21.400036    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" event={"ID":"15ae86a1-5c75-45ff-b549-32aa463d30a7","Type":"ContainerStarted","Data":"4f047f7e96974f7d4ca9335ccd17ee279e8ff8996266c20a18f9de2d348fa01e"}
Oct 30 21:57:21.509638 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:21.509582    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podStartSLOduration=2.509565217 podStartE2EDuration="2.509565217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:21.508233152 +0000 UTC m=+2697.379352280" watchObservedRunningTime="2026-10-30 21:57:21.509565217 +0000 UTC m=+2697.380684330"
Oct 30 21:57:21.635183 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:21.635121    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podStartSLOduration=2.635104719 podStartE2EDuration="2.635104719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:21.634560632 +0000 UTC m=+2697.505679751" watchObservedRunningTime="2026-10-30 21:57:21.635104719 +0000 UTC m=+2697.506223838"
Oct 30 21:57:22.185241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:22.175362    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:22.185241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:22.177149    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-6vwl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused" start-of-body=
Oct 30 21:57:22.185241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:22.177199    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused"
Oct 30 21:57:22.224976 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:22.224943    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:22.226066 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:22.226044    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8b756 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:57:22.226182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:22.226080    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused"
Oct 30 21:57:22.415087 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:57:22.415054    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:57:23.178632 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:23.178593    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-6vwl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused" start-of-body=
Oct 30 21:57:23.178901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:23.178870    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused"
Oct 30 21:57:23.224934 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:23.224909    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8b756 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:57:23.225082 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:23.224947    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused"
Oct 30 21:57:24.177641 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:24.177593    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-6vwl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused" start-of-body=
Oct 30 21:57:24.178257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:24.178227    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused"
Oct 30 21:57:24.225004 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:24.224975    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8b756 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:57:24.225146 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:24.225023    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused"
Oct 30 21:57:25.177684 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:25.177651    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-6vwl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused" start-of-body=
Oct 30 21:57:25.178448 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:25.178403    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused"
Oct 30 21:57:25.224858 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:25.224827    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8b756 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:57:25.225028 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:25.224874    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused"
Oct 30 21:57:26.175231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:26.174888    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-6vwl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused" start-of-body=
Oct 30 21:57:26.175231 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:26.174943    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused"
Oct 30 21:57:26.224524 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:26.224485    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8b756 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:57:26.224912 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:26.224535    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused"
Oct 30 21:57:27.175022 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:27.174990    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-6vwl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused" start-of-body=
Oct 30 21:57:27.175257 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:27.175045    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.220:15021/healthz/ready\": dial tcp 10.134.0.220:15021: connect: connection refused"
Oct 30 21:57:27.224603 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:27.224566    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8b756 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:57:27.224971 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:27.224628    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.221:15021/healthz/ready\": dial tcp 10.134.0.221:15021: connect: connection refused"
Oct 30 21:57:28.449861 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.449141    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"]
Oct 30 21:57:28.453897 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.453876    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.510128 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.509847    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"]
Oct 30 21:57:28.556507 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.556474    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:57:28.556683 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.556486    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:57:28.556683 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.556610    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:57:28.556957 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.556919    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:57:28.646719 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.646671    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8mhk\" (UniqueName: \"kubernetes.io/projected/f03f26b0-9253-42d4-b92d-f2254b6aba10-kube-api-access-f8mhk\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.646896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.646767    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.646896 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.646819    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.685832 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.685810    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:28.686441 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.686418    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:28.687336 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.687317    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 21:57:28.703761 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.703703    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:28.704033 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.704015    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:28.704775 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.704759    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 21:57:28.747564 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.747543    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.747670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.747582    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.747670 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.747651    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8mhk\" (UniqueName: \"kubernetes.io/projected/f03f26b0-9253-42d4-b92d-f2254b6aba10-kube-api-access-f8mhk\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.748139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.748120    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.750085 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.750066    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:28.806182 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:28.806162    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8mhk\" (UniqueName: \"kubernetes.io/projected/f03f26b0-9253-42d4-b92d-f2254b6aba10-kube-api-access-f8mhk\") pod \"maas-api-revoke-sub-2e1f5b5f-5vhk9\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:29.073139 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:29.073026    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:29.446406 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:29.446373    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"]
Oct 30 21:57:29.452038 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:57:29.452012    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03f26b0_9253_42d4_b92d_f2254b6aba10.slice/crio-7295224a003c9af2c97e2a8a1a58cc8a8eda3faa77c2f8898eaad12fff719c6f WatchSource:0}: Error finding container 7295224a003c9af2c97e2a8a1a58cc8a8eda3faa77c2f8898eaad12fff719c6f: Status 404 returned error can't find the container with id 7295224a003c9af2c97e2a8a1a58cc8a8eda3faa77c2f8898eaad12fff719c6f
Oct 30 21:57:29.459959 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:29.459933    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9" event={"ID":"f03f26b0-9253-42d4-b92d-f2254b6aba10","Type":"ContainerStarted","Data":"7295224a003c9af2c97e2a8a1a58cc8a8eda3faa77c2f8898eaad12fff719c6f"}
Oct 30 21:57:30.465491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:30.465463    5786 generic.go:356] "Generic (PLEG): container finished" podID="f03f26b0-9253-42d4-b92d-f2254b6aba10" containerID="39805d76434364b4d59fb7c306edb66e63c56f095bf03e17ee5e107dad9fad31" exitCode=0
Oct 30 21:57:30.465815 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:30.465566    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9" event={"ID":"f03f26b0-9253-42d4-b92d-f2254b6aba10","Type":"ContainerDied","Data":"39805d76434364b4d59fb7c306edb66e63c56f095bf03e17ee5e107dad9fad31"}
Oct 30 21:57:31.600879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.600854    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:31.674598 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.674574    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-xks-ca\") pod \"f03f26b0-9253-42d4-b92d-f2254b6aba10\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") "
Oct 30 21:57:31.674703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.674616    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f03f26b0-9253-42d4-b92d-f2254b6aba10-kube-api-access-f8mhk\" (UniqueName: \"kubernetes.io/projected/f03f26b0-9253-42d4-b92d-f2254b6aba10-kube-api-access-f8mhk\") pod \"f03f26b0-9253-42d4-b92d-f2254b6aba10\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") "
Oct 30 21:57:31.674703 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.674671    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-openshift-ca\") pod \"f03f26b0-9253-42d4-b92d-f2254b6aba10\" (UID: \"f03f26b0-9253-42d4-b92d-f2254b6aba10\") "
Oct 30 21:57:31.675122 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.675097    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-openshift-ca" pod "f03f26b0-9253-42d4-b92d-f2254b6aba10" (UID: "f03f26b0-9253-42d4-b92d-f2254b6aba10"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:31.676720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.676697    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-xks-ca" pod "f03f26b0-9253-42d4-b92d-f2254b6aba10" (UID: "f03f26b0-9253-42d4-b92d-f2254b6aba10"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:31.676864 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.676839    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f03f26b0-9253-42d4-b92d-f2254b6aba10-kube-api-access-f8mhk" pod "f03f26b0-9253-42d4-b92d-f2254b6aba10" (UID: "f03f26b0-9253-42d4-b92d-f2254b6aba10"). InnerVolumeSpecName "kube-api-access-f8mhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:31.776448 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.776388    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:31.776448 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.776411    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8mhk\" (UniqueName: \"kubernetes.io/projected/f03f26b0-9253-42d4-b92d-f2254b6aba10-kube-api-access-f8mhk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:31.776448 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:31.776421    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f03f26b0-9253-42d4-b92d-f2254b6aba10-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:32.476889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:32.476858    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"
Oct 30 21:57:32.476889 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:32.476881    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9" event={"ID":"f03f26b0-9253-42d4-b92d-f2254b6aba10","Type":"ContainerDied","Data":"7295224a003c9af2c97e2a8a1a58cc8a8eda3faa77c2f8898eaad12fff719c6f"}
Oct 30 21:57:32.477133 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:32.476921    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7295224a003c9af2c97e2a8a1a58cc8a8eda3faa77c2f8898eaad12fff719c6f"
Oct 30 21:57:40.996530 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:40.996499    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"]
Oct 30 21:57:40.996907 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:40.996899    5786 cpu_manager.go:414] "containerMap: removing container" podUID="f03f26b0-9253-42d4-b92d-f2254b6aba10" containerName="revoke-keys"
Oct 30 21:57:40.996951 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:40.996910    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f03f26b0-9253-42d4-b92d-f2254b6aba10" containerName="revoke-keys"
Oct 30 21:57:40.997005 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:40.996994    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="f03f26b0-9253-42d4-b92d-f2254b6aba10" containerName="revoke-keys"
Oct 30 21:57:41.000277 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.000256    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.051965 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.051941    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:57:41.059476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.059448    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.059611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.059495    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m59xm\" (UniqueName: \"kubernetes.io/projected/146cc47e-d072-49ca-acd6-8d85fa686f7c-kube-api-access-m59xm\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.059611 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.059591    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.101144 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.101114    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:57:41.101293 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.101156    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:57:41.101293 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.101241    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:57:41.103160 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.103144    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"]
Oct 30 21:57:41.160495 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.160471    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.160601 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.160509    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m59xm\" (UniqueName: \"kubernetes.io/projected/146cc47e-d072-49ca-acd6-8d85fa686f7c-kube-api-access-m59xm\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.160651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.160633    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.161103 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.161086    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.162936 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.162913    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.216931 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.216912    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m59xm\" (UniqueName: \"kubernetes.io/projected/146cc47e-d072-49ca-acd6-8d85fa686f7c-kube-api-access-m59xm\") pod \"maas-api-revoke-sub-ec843018-t5jfl\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.313755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.313700    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:41.724537 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:41.724491    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"]
Oct 30 21:57:41.728046 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:57:41.728013    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod146cc47e_d072_49ca_acd6_8d85fa686f7c.slice/crio-a9eb51c0008f5be350aa072794daa0b5a0a2cafbf15b27d3ae83b471e2896670 WatchSource:0}: Error finding container a9eb51c0008f5be350aa072794daa0b5a0a2cafbf15b27d3ae83b471e2896670: Status 404 returned error can't find the container with id a9eb51c0008f5be350aa072794daa0b5a0a2cafbf15b27d3ae83b471e2896670
Oct 30 21:57:42.529111 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:42.529075    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl" event={"ID":"146cc47e-d072-49ca-acd6-8d85fa686f7c","Type":"ContainerStarted","Data":"38ea0238707eb652740bbb41df01dcf818a2da392310a93144f1db7c2029750e"}
Oct 30 21:57:42.529534 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:42.529124    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl" event={"ID":"146cc47e-d072-49ca-acd6-8d85fa686f7c","Type":"ContainerStarted","Data":"a9eb51c0008f5be350aa072794daa0b5a0a2cafbf15b27d3ae83b471e2896670"}
Oct 30 21:57:42.647330 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:42.646964    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl" podStartSLOduration=2.646946279 podStartE2EDuration="2.646946279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:42.644309961 +0000 UTC m=+2718.515429074" watchObservedRunningTime="2026-10-30 21:57:42.646946279 +0000 UTC m=+2718.518065399"
Oct 30 21:57:43.535186 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:43.535149    5786 generic.go:356] "Generic (PLEG): container finished" podID="146cc47e-d072-49ca-acd6-8d85fa686f7c" containerID="38ea0238707eb652740bbb41df01dcf818a2da392310a93144f1db7c2029750e" exitCode=0
Oct 30 21:57:43.535623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:43.535238    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl" event={"ID":"146cc47e-d072-49ca-acd6-8d85fa686f7c","Type":"ContainerDied","Data":"38ea0238707eb652740bbb41df01dcf818a2da392310a93144f1db7c2029750e"}
Oct 30 21:57:44.671677 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.671651    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:44.800549 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.800492    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-openshift-ca\") pod \"146cc47e-d072-49ca-acd6-8d85fa686f7c\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") "
Oct 30 21:57:44.800691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.800611    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-xks-ca\") pod \"146cc47e-d072-49ca-acd6-8d85fa686f7c\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") "
Oct 30 21:57:44.800691 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.800666    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/146cc47e-d072-49ca-acd6-8d85fa686f7c-kube-api-access-m59xm\" (UniqueName: \"kubernetes.io/projected/146cc47e-d072-49ca-acd6-8d85fa686f7c-kube-api-access-m59xm\") pod \"146cc47e-d072-49ca-acd6-8d85fa686f7c\" (UID: \"146cc47e-d072-49ca-acd6-8d85fa686f7c\") "
Oct 30 21:57:44.800880 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.800856    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-openshift-ca" pod "146cc47e-d072-49ca-acd6-8d85fa686f7c" (UID: "146cc47e-d072-49ca-acd6-8d85fa686f7c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:44.802599 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.802575    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-xks-ca" pod "146cc47e-d072-49ca-acd6-8d85fa686f7c" (UID: "146cc47e-d072-49ca-acd6-8d85fa686f7c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:44.802712 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.802691    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/146cc47e-d072-49ca-acd6-8d85fa686f7c-kube-api-access-m59xm" pod "146cc47e-d072-49ca-acd6-8d85fa686f7c" (UID: "146cc47e-d072-49ca-acd6-8d85fa686f7c"). InnerVolumeSpecName "kube-api-access-m59xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:44.902246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.902222    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:44.902246 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.902245    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m59xm\" (UniqueName: \"kubernetes.io/projected/146cc47e-d072-49ca-acd6-8d85fa686f7c-kube-api-access-m59xm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:44.902383 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:44.902257    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/146cc47e-d072-49ca-acd6-8d85fa686f7c-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:45.546604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:45.546575    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"
Oct 30 21:57:45.546604 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:45.546585    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl" event={"ID":"146cc47e-d072-49ca-acd6-8d85fa686f7c","Type":"ContainerDied","Data":"a9eb51c0008f5be350aa072794daa0b5a0a2cafbf15b27d3ae83b471e2896670"}
Oct 30 21:57:45.546820 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:57:45.546626    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9eb51c0008f5be350aa072794daa0b5a0a2cafbf15b27d3ae83b471e2896670"
Oct 30 21:57:52.353379 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:57:52.353342    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:58:22.352332 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:58:22.352293    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:58:26.141847 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.141817    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"]
Oct 30 21:58:26.142396 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.142380    5786 cpu_manager.go:414] "containerMap: removing container" podUID="146cc47e-d072-49ca-acd6-8d85fa686f7c" containerName="revoke-keys"
Oct 30 21:58:26.142471 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.142398    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="146cc47e-d072-49ca-acd6-8d85fa686f7c" containerName="revoke-keys"
Oct 30 21:58:26.142529 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.142494    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="146cc47e-d072-49ca-acd6-8d85fa686f7c" containerName="revoke-keys"
Oct 30 21:58:26.145623 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.145597    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.198158 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.198130    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:58:26.198338 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.198184    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:58:26.223648 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.223621    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.223776 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.223676    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.223818 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.223782    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrwwp\" (UniqueName: \"kubernetes.io/projected/ff7a0fad-5eb9-4576-b480-566649a67864-kube-api-access-zrwwp\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.247232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.247193    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:58:26.247232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.247193    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:58:26.250084 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.250060    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"]
Oct 30 21:58:26.324596 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.324573    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.324705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.324605    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrwwp\" (UniqueName: \"kubernetes.io/projected/ff7a0fad-5eb9-4576-b480-566649a67864-kube-api-access-zrwwp\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.324705 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.324680    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.325163 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.325135    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.326991 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.326972    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.381513 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.381488    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrwwp\" (UniqueName: \"kubernetes.io/projected/ff7a0fad-5eb9-4576-b480-566649a67864-kube-api-access-zrwwp\") pod \"maas-api-revoke-sub-046e1343-qkpqz\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.462560 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.462538    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:26.833823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:26.833637    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"]
Oct 30 21:58:26.836669 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:58:26.836641    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7a0fad_5eb9_4576_b480_566649a67864.slice/crio-a968d5c258b5a7e8b621dc8458063c7d5f9ea280e27679443bf519eb22fa6556 WatchSource:0}: Error finding container a968d5c258b5a7e8b621dc8458063c7d5f9ea280e27679443bf519eb22fa6556: Status 404 returned error can't find the container with id a968d5c258b5a7e8b621dc8458063c7d5f9ea280e27679443bf519eb22fa6556
Oct 30 21:58:27.593050 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:27.593021    5786 scope.go:122] "RemoveContainer" containerID="02784a96f9bec834d841146a02a1d39a24f3145420c0d5db2d8149d96e0b7c37"
Oct 30 21:58:27.765002 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:27.764976    5786 generic.go:356] "Generic (PLEG): container finished" podID="ff7a0fad-5eb9-4576-b480-566649a67864" containerID="a9d4366ac07b2cead4cd23f367941fcd99501ccd2e3f294818a7629b52ae084d" exitCode=0
Oct 30 21:58:27.765129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:27.765036    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz" event={"ID":"ff7a0fad-5eb9-4576-b480-566649a67864","Type":"ContainerDied","Data":"a9d4366ac07b2cead4cd23f367941fcd99501ccd2e3f294818a7629b52ae084d"}
Oct 30 21:58:27.765129 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:27.765085    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz" event={"ID":"ff7a0fad-5eb9-4576-b480-566649a67864","Type":"ContainerStarted","Data":"a968d5c258b5a7e8b621dc8458063c7d5f9ea280e27679443bf519eb22fa6556"}
Oct 30 21:58:28.900542 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:28.900520    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:28.948767 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:28.948743    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-xks-ca\") pod \"ff7a0fad-5eb9-4576-b480-566649a67864\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") "
Oct 30 21:58:28.948879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:28.948783    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff7a0fad-5eb9-4576-b480-566649a67864-kube-api-access-zrwwp\" (UniqueName: \"kubernetes.io/projected/ff7a0fad-5eb9-4576-b480-566649a67864-kube-api-access-zrwwp\") pod \"ff7a0fad-5eb9-4576-b480-566649a67864\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") "
Oct 30 21:58:28.948879 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:28.948810    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-openshift-ca\") pod \"ff7a0fad-5eb9-4576-b480-566649a67864\" (UID: \"ff7a0fad-5eb9-4576-b480-566649a67864\") "
Oct 30 21:58:28.949197 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:28.949171    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-openshift-ca" pod "ff7a0fad-5eb9-4576-b480-566649a67864" (UID: "ff7a0fad-5eb9-4576-b480-566649a67864"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:28.950811 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:28.950789    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-xks-ca" pod "ff7a0fad-5eb9-4576-b480-566649a67864" (UID: "ff7a0fad-5eb9-4576-b480-566649a67864"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:28.950898 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:28.950859    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7a0fad-5eb9-4576-b480-566649a67864-kube-api-access-zrwwp" pod "ff7a0fad-5eb9-4576-b480-566649a67864" (UID: "ff7a0fad-5eb9-4576-b480-566649a67864"). InnerVolumeSpecName "kube-api-access-zrwwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:29.050064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:29.050007    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:29.050064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:29.050032    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrwwp\" (UniqueName: \"kubernetes.io/projected/ff7a0fad-5eb9-4576-b480-566649a67864-kube-api-access-zrwwp\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:29.050064 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:29.050041    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff7a0fad-5eb9-4576-b480-566649a67864-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:29.777013 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:29.776990    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"
Oct 30 21:58:29.777165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:29.777032    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz" event={"ID":"ff7a0fad-5eb9-4576-b480-566649a67864","Type":"ContainerDied","Data":"a968d5c258b5a7e8b621dc8458063c7d5f9ea280e27679443bf519eb22fa6556"}
Oct 30 21:58:29.777165 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:29.777070    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a968d5c258b5a7e8b621dc8458063c7d5f9ea280e27679443bf519eb22fa6556"
Oct 30 21:58:36.894261 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:36.894230    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"]
Oct 30 21:58:36.894984 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:36.894964    5786 cpu_manager.go:414] "containerMap: removing container" podUID="ff7a0fad-5eb9-4576-b480-566649a67864" containerName="revoke-keys"
Oct 30 21:58:36.895080 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:36.894988    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff7a0fad-5eb9-4576-b480-566649a67864" containerName="revoke-keys"
Oct 30 21:58:36.895120 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:36.895084    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff7a0fad-5eb9-4576-b480-566649a67864" containerName="revoke-keys"
Oct 30 21:58:36.898799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:36.898781    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.000027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.000000    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:58:37.000027 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.000012    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:58:37.000320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.000150    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:58:37.000320 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.000238    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:58:37.002011 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.001985    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"]
Oct 30 21:58:37.016550 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.016530    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.016653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.016605    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7q9j\" (UniqueName: \"kubernetes.io/projected/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-kube-api-access-g7q9j\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.016653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.016634    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.118093 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.118073    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.118232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.118124    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7q9j\" (UniqueName: \"kubernetes.io/projected/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-kube-api-access-g7q9j\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.118232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.118143    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.118755 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.118731    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.120561 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.120537    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.178619 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.178597    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7q9j\" (UniqueName: \"kubernetes.io/projected/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-kube-api-access-g7q9j\") pod \"maas-api-revoke-sub-d3fba056-4hl4k\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.211444 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.211425    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:37.585759 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.585736    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"]
Oct 30 21:58:37.587933 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:58:37.587910    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbbc227_5016_4fb3_a5c4_f476cc7ca626.slice/crio-80ca2ab8713eee5556aef2a342fde9c5d0b48553688e13ffa9975d5858c20051 WatchSource:0}: Error finding container 80ca2ab8713eee5556aef2a342fde9c5d0b48553688e13ffa9975d5858c20051: Status 404 returned error can't find the container with id 80ca2ab8713eee5556aef2a342fde9c5d0b48553688e13ffa9975d5858c20051
Oct 30 21:58:37.817762 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:37.817735    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k" event={"ID":"7dbbc227-5016-4fb3-a5c4-f476cc7ca626","Type":"ContainerStarted","Data":"80ca2ab8713eee5556aef2a342fde9c5d0b48553688e13ffa9975d5858c20051"}
Oct 30 21:58:38.832826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:38.832747    5786 generic.go:356] "Generic (PLEG): container finished" podID="7dbbc227-5016-4fb3-a5c4-f476cc7ca626" containerID="ca66245c268a46cf9a6fcde9d054546b686bf9b057e2b2767c4d48f761f604a0" exitCode=0
Oct 30 21:58:38.832826 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:38.832810    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k" event={"ID":"7dbbc227-5016-4fb3-a5c4-f476cc7ca626","Type":"ContainerDied","Data":"ca66245c268a46cf9a6fcde9d054546b686bf9b057e2b2767c4d48f761f604a0"}
Oct 30 21:58:40.033979 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.033950    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:40.154771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.153854    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-kube-api-access-g7q9j\" (UniqueName: \"kubernetes.io/projected/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-kube-api-access-g7q9j\") pod \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") "
Oct 30 21:58:40.154771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.153936    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-openshift-ca\") pod \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") "
Oct 30 21:58:40.154771 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.154008    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-xks-ca\") pod \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\" (UID: \"7dbbc227-5016-4fb3-a5c4-f476cc7ca626\") "
Oct 30 21:58:40.163640 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.163580    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-xks-ca" pod "7dbbc227-5016-4fb3-a5c4-f476cc7ca626" (UID: "7dbbc227-5016-4fb3-a5c4-f476cc7ca626"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:40.179238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.178335    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-openshift-ca" pod "7dbbc227-5016-4fb3-a5c4-f476cc7ca626" (UID: "7dbbc227-5016-4fb3-a5c4-f476cc7ca626"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:40.198823 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.198762    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-kube-api-access-g7q9j" pod "7dbbc227-5016-4fb3-a5c4-f476cc7ca626" (UID: "7dbbc227-5016-4fb3-a5c4-f476cc7ca626"). InnerVolumeSpecName "kube-api-access-g7q9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:40.260241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.255862    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7q9j\" (UniqueName: \"kubernetes.io/projected/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-kube-api-access-g7q9j\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:40.260241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.255898    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:40.260241 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.255916    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7dbbc227-5016-4fb3-a5c4-f476cc7ca626-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:40.853292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.852788    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k" event={"ID":"7dbbc227-5016-4fb3-a5c4-f476cc7ca626","Type":"ContainerDied","Data":"80ca2ab8713eee5556aef2a342fde9c5d0b48553688e13ffa9975d5858c20051"}
Oct 30 21:58:40.853292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.852848    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80ca2ab8713eee5556aef2a342fde9c5d0b48553688e13ffa9975d5858c20051"
Oct 30 21:58:40.853292 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:40.852917    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"
Oct 30 21:58:52.354522 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:58:52.354487    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:58:59.025420 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:59.025300    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:58:59.026219 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:58:59.026156    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:59:14.560491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.560453    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"]
Oct 30 21:59:14.560901 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.560886    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7dbbc227-5016-4fb3-a5c4-f476cc7ca626" containerName="revoke-keys"
Oct 30 21:59:14.560945 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.560904    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7dbbc227-5016-4fb3-a5c4-f476cc7ca626" containerName="revoke-keys"
Oct 30 21:59:14.561003 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.560995    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7dbbc227-5016-4fb3-a5c4-f476cc7ca626" containerName="revoke-keys"
Oct 30 21:59:14.564238 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.564219    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.615751 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.615727    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:59:14.629553 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.629535    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:59:14.658217 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.658173    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.658373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.658307    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.658373 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.658347    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnpwj\" (UniqueName: \"kubernetes.io/projected/7462440c-9ee9-47dd-b623-ff3351aad152-kube-api-access-fnpwj\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.665135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.665114    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:59:14.665135 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.665129    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:59:14.667043 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.667027    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"]
Oct 30 21:59:14.758887 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.758860    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.759007 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.758907    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.759007 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.758925    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnpwj\" (UniqueName: \"kubernetes.io/projected/7462440c-9ee9-47dd-b623-ff3351aad152-kube-api-access-fnpwj\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.759479 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.759461    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.761296 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.761279    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.815653 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.815607    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnpwj\" (UniqueName: \"kubernetes.io/projected/7462440c-9ee9-47dd-b623-ff3351aad152-kube-api-access-fnpwj\") pod \"maas-api-revoke-sub-ae594979-4zbtb\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:14.880756 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:14.880737    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:15.258627 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:15.258600    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"]
Oct 30 21:59:15.262159 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:59:15.262129    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7462440c_9ee9_47dd_b623_ff3351aad152.slice/crio-779d70493365a6905d08d55508ad17993b81650169e35fbe607e4594898099b4 WatchSource:0}: Error finding container 779d70493365a6905d08d55508ad17993b81650169e35fbe607e4594898099b4: Status 404 returned error can't find the container with id 779d70493365a6905d08d55508ad17993b81650169e35fbe607e4594898099b4
Oct 30 21:59:16.047536 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:16.047507    5786 generic.go:356] "Generic (PLEG): container finished" podID="7462440c-9ee9-47dd-b623-ff3351aad152" containerID="e2eea57fe98d9794a101e893dd469d1dc1130b941724a5776f3cd191e1d700f0" exitCode=0
Oct 30 21:59:16.047929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:16.047590    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb" event={"ID":"7462440c-9ee9-47dd-b623-ff3351aad152","Type":"ContainerDied","Data":"e2eea57fe98d9794a101e893dd469d1dc1130b941724a5776f3cd191e1d700f0"}
Oct 30 21:59:16.047929 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:16.047638    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb" event={"ID":"7462440c-9ee9-47dd-b623-ff3351aad152","Type":"ContainerStarted","Data":"779d70493365a6905d08d55508ad17993b81650169e35fbe607e4594898099b4"}
Oct 30 21:59:17.180639 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.180612    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:17.280886 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.280858    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7462440c-9ee9-47dd-b623-ff3351aad152-kube-api-access-fnpwj\" (UniqueName: \"kubernetes.io/projected/7462440c-9ee9-47dd-b623-ff3351aad152-kube-api-access-fnpwj\") pod \"7462440c-9ee9-47dd-b623-ff3351aad152\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") "
Oct 30 21:59:17.281018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.280922    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-openshift-ca\") pod \"7462440c-9ee9-47dd-b623-ff3351aad152\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") "
Oct 30 21:59:17.281018 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.280963    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-xks-ca\") pod \"7462440c-9ee9-47dd-b623-ff3351aad152\" (UID: \"7462440c-9ee9-47dd-b623-ff3351aad152\") "
Oct 30 21:59:17.281289 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.281265    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-openshift-ca" pod "7462440c-9ee9-47dd-b623-ff3351aad152" (UID: "7462440c-9ee9-47dd-b623-ff3351aad152"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:17.282903 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.282887    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7462440c-9ee9-47dd-b623-ff3351aad152-kube-api-access-fnpwj" pod "7462440c-9ee9-47dd-b623-ff3351aad152" (UID: "7462440c-9ee9-47dd-b623-ff3351aad152"). InnerVolumeSpecName "kube-api-access-fnpwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:17.283073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.283046    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-xks-ca" pod "7462440c-9ee9-47dd-b623-ff3351aad152" (UID: "7462440c-9ee9-47dd-b623-ff3351aad152"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:17.382813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.382744    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnpwj\" (UniqueName: \"kubernetes.io/projected/7462440c-9ee9-47dd-b623-ff3351aad152-kube-api-access-fnpwj\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:17.382813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.382789    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:17.382813 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:17.382812    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7462440c-9ee9-47dd-b623-ff3351aad152-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:18.058401 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:18.058377    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"
Oct 30 21:59:18.058590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:18.058415    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb" event={"ID":"7462440c-9ee9-47dd-b623-ff3351aad152","Type":"ContainerDied","Data":"779d70493365a6905d08d55508ad17993b81650169e35fbe607e4594898099b4"}
Oct 30 21:59:18.058590 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:18.058465    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="779d70493365a6905d08d55508ad17993b81650169e35fbe607e4594898099b4"
Oct 30 21:59:22.353723 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:59:22.353691    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:59:49.072278 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:49.072248    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"]
Oct 30 21:59:49.121688 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:49.121659    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e62e732-fml8m"]
Oct 30 21:59:50.358476 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:50.358448    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1524c15-b9d0-460d-86ea-a7e5c1f566bd" path="/var/lib/kubelet/pods/d1524c15-b9d0-460d-86ea-a7e5c1f566bd/volumes"
Oct 30 21:59:52.352686 ip-10-0-132-134 kubenswrapper[5786]: E1030 21:59:52.352650    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 21:59:52.567327 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.567294    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"]
Oct 30 21:59:52.567720 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.567708    5786 cpu_manager.go:414] "containerMap: removing container" podUID="7462440c-9ee9-47dd-b623-ff3351aad152" containerName="revoke-keys"
Oct 30 21:59:52.567764 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.567721    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7462440c-9ee9-47dd-b623-ff3351aad152" containerName="revoke-keys"
Oct 30 21:59:52.567799 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.567780    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="7462440c-9ee9-47dd-b623-ff3351aad152" containerName="revoke-keys"
Oct 30 21:59:52.570674 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.570659    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.671702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.671669    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:59:52.671702 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.671684    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 21:59:52.671923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.671707    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:59:52.671923 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.671723    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:59:52.674651 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.674623    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"]
Oct 30 21:59:52.710807 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.710786    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.710925 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.710813    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctjmh\" (UniqueName: \"kubernetes.io/projected/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-kube-api-access-ctjmh\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.710968 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.710944    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.812044 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.812009    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.812177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.812105    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.812177 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.812122    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctjmh\" (UniqueName: \"kubernetes.io/projected/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-kube-api-access-ctjmh\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.812658 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.812628    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.814491 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.814469    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.868102 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.868074    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctjmh\" (UniqueName: \"kubernetes.io/projected/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-kube-api-access-ctjmh\") pod \"maas-api-revoke-sub-a4aef7d7-p4r9x\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:52.882423 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:52.882403    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:53.252235 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:53.252162    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"]
Oct 30 21:59:53.255131 ip-10-0-132-134 kubenswrapper[5786]: W1030 21:59:53.255102    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e9dcc0_92b0_49ee_8f4f_a862632036c4.slice/crio-bd25cf72cbec8be7bab226c08c548185814359acd96f141ae8ed2797c97543cb WatchSource:0}: Error finding container bd25cf72cbec8be7bab226c08c548185814359acd96f141ae8ed2797c97543cb: Status 404 returned error can't find the container with id bd25cf72cbec8be7bab226c08c548185814359acd96f141ae8ed2797c97543cb
Oct 30 21:59:54.243770 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:54.243741    5786 generic.go:356] "Generic (PLEG): container finished" podID="e3e9dcc0-92b0-49ee-8f4f-a862632036c4" containerID="12da0f305b968877a7423b48c0b6b42c7b55d2f1d53388bae86cc77a0ad9129c" exitCode=0
Oct 30 21:59:54.244100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:54.243822    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x" event={"ID":"e3e9dcc0-92b0-49ee-8f4f-a862632036c4","Type":"ContainerDied","Data":"12da0f305b968877a7423b48c0b6b42c7b55d2f1d53388bae86cc77a0ad9129c"}
Oct 30 21:59:54.244100 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:54.243867    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x" event={"ID":"e3e9dcc0-92b0-49ee-8f4f-a862632036c4","Type":"ContainerStarted","Data":"bd25cf72cbec8be7bab226c08c548185814359acd96f141ae8ed2797c97543cb"}
Oct 30 21:59:55.380437 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.380418    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:55.537397 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.537340    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-xks-ca\") pod \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") "
Oct 30 21:59:55.537500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.537452    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-openshift-ca\") pod \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") "
Oct 30 21:59:55.537500 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.537478    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-kube-api-access-ctjmh\" (UniqueName: \"kubernetes.io/projected/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-kube-api-access-ctjmh\") pod \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\" (UID: \"e3e9dcc0-92b0-49ee-8f4f-a862632036c4\") "
Oct 30 21:59:55.537745 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.537727    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-openshift-ca" pod "e3e9dcc0-92b0-49ee-8f4f-a862632036c4" (UID: "e3e9dcc0-92b0-49ee-8f4f-a862632036c4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:55.539613 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.539589    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-xks-ca" pod "e3e9dcc0-92b0-49ee-8f4f-a862632036c4" (UID: "e3e9dcc0-92b0-49ee-8f4f-a862632036c4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:55.539708 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.539606    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-kube-api-access-ctjmh" pod "e3e9dcc0-92b0-49ee-8f4f-a862632036c4" (UID: "e3e9dcc0-92b0-49ee-8f4f-a862632036c4"). InnerVolumeSpecName "kube-api-access-ctjmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:55.638073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.638050    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:55.638073 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.638071    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctjmh\" (UniqueName: \"kubernetes.io/projected/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-kube-api-access-ctjmh\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:55.638232 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:55.638080    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e9dcc0-92b0-49ee-8f4f-a862632036c4-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:56.254298 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:56.254273    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"
Oct 30 21:59:56.254421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:56.254304    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x" event={"ID":"e3e9dcc0-92b0-49ee-8f4f-a862632036c4","Type":"ContainerDied","Data":"bd25cf72cbec8be7bab226c08c548185814359acd96f141ae8ed2797c97543cb"}
Oct 30 21:59:56.254421 ip-10-0-132-134 kubenswrapper[5786]: I1030 21:59:56.254348    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd25cf72cbec8be7bab226c08c548185814359acd96f141ae8ed2797c97543cb"
Oct 30 22:00:00.159347 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.159316    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"]
Oct 30 22:00:00.159723 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.159714    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e3e9dcc0-92b0-49ee-8f4f-a862632036c4" containerName="revoke-keys"
Oct 30 22:00:00.159766 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.159724    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3e9dcc0-92b0-49ee-8f4f-a862632036c4" containerName="revoke-keys"
Oct 30 22:00:00.159820 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.159810    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3e9dcc0-92b0-49ee-8f4f-a862632036c4" containerName="revoke-keys"
Oct 30 22:00:00.162996 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.162978    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:00.214800 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.214768    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:00:00.214800 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.214795    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:00:00.264312 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.264284    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:00:00.266113 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.266091    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"]
Oct 30 22:00:00.276794 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.276756    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqjt7\" (UniqueName: \"kubernetes.io/projected/82acf44a-fdee-485d-a483-42749465d267-kube-api-access-wqjt7\") pod \"maas-api-key-cleanup-29889960-lkvzh\" (UID: \"82acf44a-fdee-485d-a483-42749465d267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:00.276940 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.276875    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acf44a-fdee-485d-a483-42749465d267-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889960-lkvzh\" (UID: \"82acf44a-fdee-485d-a483-42749465d267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:00.378035 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.378010    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqjt7\" (UniqueName: \"kubernetes.io/projected/82acf44a-fdee-485d-a483-42749465d267-kube-api-access-wqjt7\") pod \"maas-api-key-cleanup-29889960-lkvzh\" (UID: \"82acf44a-fdee-485d-a483-42749465d267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:00.378150 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.378076    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acf44a-fdee-485d-a483-42749465d267-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889960-lkvzh\" (UID: \"82acf44a-fdee-485d-a483-42749465d267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:00.378669 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.378649    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acf44a-fdee-485d-a483-42749465d267-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889960-lkvzh\" (UID: \"82acf44a-fdee-485d-a483-42749465d267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:00.436028 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.436006    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqjt7\" (UniqueName: \"kubernetes.io/projected/82acf44a-fdee-485d-a483-42749465d267-kube-api-access-wqjt7\") pod \"maas-api-key-cleanup-29889960-lkvzh\" (UID: \"82acf44a-fdee-485d-a483-42749465d267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:00.475627 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.475607    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:00.848409 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:00.848358    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"]
Oct 30 22:00:00.850738 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:00:00.850714    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82acf44a_fdee_485d_a483_42749465d267.slice/crio-fdf34f2783a3132c3781f1baeb0f77b36adea82f1acf45862c1a0739a4be56d0 WatchSource:0}: Error finding container fdf34f2783a3132c3781f1baeb0f77b36adea82f1acf45862c1a0739a4be56d0: Status 404 returned error can't find the container with id fdf34f2783a3132c3781f1baeb0f77b36adea82f1acf45862c1a0739a4be56d0
Oct 30 22:00:01.279424 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:01.279396    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" event={"ID":"82acf44a-fdee-485d-a483-42749465d267","Type":"ContainerStarted","Data":"d2cf2adf29db70382f41ca05463645f335037df67b2c252b1b4dbfb93c2c5755"}
Oct 30 22:00:01.279787 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:01.279455    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" event={"ID":"82acf44a-fdee-485d-a483-42749465d267","Type":"ContainerStarted","Data":"fdf34f2783a3132c3781f1baeb0f77b36adea82f1acf45862c1a0739a4be56d0"}
Oct 30 22:00:01.392620 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:01.392580    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" podStartSLOduration=1.392562841 podStartE2EDuration="1.392562841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:01.392458808 +0000 UTC m=+2857.263577927" watchObservedRunningTime="2026-10-30 22:00:01.392562841 +0000 UTC m=+2857.263682030"
Oct 30 22:00:02.285954 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:02.285918    5786 generic.go:356] "Generic (PLEG): container finished" podID="82acf44a-fdee-485d-a483-42749465d267" containerID="d2cf2adf29db70382f41ca05463645f335037df67b2c252b1b4dbfb93c2c5755" exitCode=60
Oct 30 22:00:02.286445 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:02.286006    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" event={"ID":"82acf44a-fdee-485d-a483-42749465d267","Type":"ContainerDied","Data":"d2cf2adf29db70382f41ca05463645f335037df67b2c252b1b4dbfb93c2c5755"}
Oct 30 22:00:02.286445 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:02.286244    5786 scope.go:122] "RemoveContainer" containerID="d2cf2adf29db70382f41ca05463645f335037df67b2c252b1b4dbfb93c2c5755"
Oct 30 22:00:03.292433 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:03.292364    5786 generic.go:356] "Generic (PLEG): container finished" podID="82acf44a-fdee-485d-a483-42749465d267" containerID="a180bd7fb69a0d56c5e49762e14a6ed2d9afd6ff3b8f98acb9ad6d3541677dc6" exitCode=60
Oct 30 22:00:03.292846 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:03.292448    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" event={"ID":"82acf44a-fdee-485d-a483-42749465d267","Type":"ContainerDied","Data":"a180bd7fb69a0d56c5e49762e14a6ed2d9afd6ff3b8f98acb9ad6d3541677dc6"}
Oct 30 22:00:03.292846 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:03.292510    5786 scope.go:122] "RemoveContainer" containerID="d2cf2adf29db70382f41ca05463645f335037df67b2c252b1b4dbfb93c2c5755"
Oct 30 22:00:03.292846 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:03.292746    5786 scope.go:122] "RemoveContainer" containerID="a180bd7fb69a0d56c5e49762e14a6ed2d9afd6ff3b8f98acb9ad6d3541677dc6"
Oct 30 22:00:03.292980 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:00:03.292946    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889960-lkvzh_odh-ai-gateway-infra(82acf44a-fdee-485d-a483-42749465d267)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" podUID="82acf44a-fdee-485d-a483-42749465d267"
Oct 30 22:00:04.298873 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:04.298843    5786 scope.go:122] "RemoveContainer" containerID="a180bd7fb69a0d56c5e49762e14a6ed2d9afd6ff3b8f98acb9ad6d3541677dc6"
Oct 30 22:00:04.299375 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:00:04.299087    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889960-lkvzh_odh-ai-gateway-infra(82acf44a-fdee-485d-a483-42749465d267)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" podUID="82acf44a-fdee-485d-a483-42749465d267"
Oct 30 22:00:14.026785 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:14.026296    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:00:14.026785 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:14.026371    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:14.730238 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:14.727342    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-6vwl6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.220:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:00:14.730238 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:14.727389    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.220:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:16.352185 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:16.352157    5786 scope.go:122] "RemoveContainer" containerID="a180bd7fb69a0d56c5e49762e14a6ed2d9afd6ff3b8f98acb9ad6d3541677dc6"
Oct 30 22:00:17.391918 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:17.391846    5786 generic.go:356] "Generic (PLEG): container finished" podID="82acf44a-fdee-485d-a483-42749465d267" containerID="bdf728eaac3bac2fc6548564fef33efb248bd41d7392320554e5127a83a8cbc6" exitCode=60
Oct 30 22:00:17.392391 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:17.391915    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" event={"ID":"82acf44a-fdee-485d-a483-42749465d267","Type":"ContainerDied","Data":"bdf728eaac3bac2fc6548564fef33efb248bd41d7392320554e5127a83a8cbc6"}
Oct 30 22:00:17.392391 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:17.391972    5786 scope.go:122] "RemoveContainer" containerID="a180bd7fb69a0d56c5e49762e14a6ed2d9afd6ff3b8f98acb9ad6d3541677dc6"
Oct 30 22:00:17.392391 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:17.392302    5786 scope.go:122] "RemoveContainer" containerID="bdf728eaac3bac2fc6548564fef33efb248bd41d7392320554e5127a83a8cbc6"
Oct 30 22:00:17.392569 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:00:17.392541    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889960-lkvzh_odh-ai-gateway-infra(82acf44a-fdee-485d-a483-42749465d267)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" podUID="82acf44a-fdee-485d-a483-42749465d267"
Oct 30 22:00:18.522469 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:18.522431    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"]
Oct 30 22:00:18.657396 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:18.657372    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:18.681651 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:18.681629    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82acf44a-fdee-485d-a483-42749465d267-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acf44a-fdee-485d-a483-42749465d267-maas-api-service-ca\") pod \"82acf44a-fdee-485d-a483-42749465d267\" (UID: \"82acf44a-fdee-485d-a483-42749465d267\") "
Oct 30 22:00:18.681794 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:18.681782    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82acf44a-fdee-485d-a483-42749465d267-kube-api-access-wqjt7\" (UniqueName: \"kubernetes.io/projected/82acf44a-fdee-485d-a483-42749465d267-kube-api-access-wqjt7\") pod \"82acf44a-fdee-485d-a483-42749465d267\" (UID: \"82acf44a-fdee-485d-a483-42749465d267\") "
Oct 30 22:00:18.681968 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:18.681948    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82acf44a-fdee-485d-a483-42749465d267-maas-api-service-ca" pod "82acf44a-fdee-485d-a483-42749465d267" (UID: "82acf44a-fdee-485d-a483-42749465d267"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:18.683830 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:18.683799    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82acf44a-fdee-485d-a483-42749465d267-kube-api-access-wqjt7" pod "82acf44a-fdee-485d-a483-42749465d267" (UID: "82acf44a-fdee-485d-a483-42749465d267"). InnerVolumeSpecName "kube-api-access-wqjt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:18.782519 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:18.782467    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqjt7\" (UniqueName: \"kubernetes.io/projected/82acf44a-fdee-485d-a483-42749465d267-kube-api-access-wqjt7\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:18.782519 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:18.782494    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acf44a-fdee-485d-a483-42749465d267-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:19.403896 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:19.403867    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"
Oct 30 22:00:19.404107 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:19.403872    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh" event={"ID":"82acf44a-fdee-485d-a483-42749465d267","Type":"ContainerDied","Data":"fdf34f2783a3132c3781f1baeb0f77b36adea82f1acf45862c1a0739a4be56d0"}
Oct 30 22:00:19.404107 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:19.403992    5786 scope.go:122] "RemoveContainer" containerID="bdf728eaac3bac2fc6548564fef33efb248bd41d7392320554e5127a83a8cbc6"
Oct 30 22:00:19.564806 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:19.564782    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"]
Oct 30 22:00:19.568475 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:19.568453    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-lkvzh"]
Oct 30 22:00:20.357766 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:20.357735    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82acf44a-fdee-485d-a483-42749465d267" path="/var/lib/kubelet/pods/82acf44a-fdee-485d-a483-42749465d267/volumes"
Oct 30 22:00:22.353695 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:00:22.353663    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:00:23.790907 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.790876    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"]
Oct 30 22:00:23.791370 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.791355    5786 cpu_manager.go:414] "containerMap: removing container" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:00:23.791424 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.791371    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:00:23.791424 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.791383    5786 cpu_manager.go:414] "containerMap: removing container" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:00:23.791424 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.791388    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:00:23.791562 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.791470    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:00:23.791562 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.791479    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:00:23.795771 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.795756    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.825048 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.825022    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.825146 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.825058    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.825146 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.825077    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99lx6\" (UniqueName: \"kubernetes.io/projected/e5c71e6e-8806-4884-8329-987e224d23b2-kube-api-access-99lx6\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.848968 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.848944    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:00:23.849043 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.849023    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:00:23.898493 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.898466    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:00:23.898612 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.898468    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:00:23.900331 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.900313    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"]
Oct 30 22:00:23.926448 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.926421    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99lx6\" (UniqueName: \"kubernetes.io/projected/e5c71e6e-8806-4884-8329-987e224d23b2-kube-api-access-99lx6\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.926572 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.926557    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.926640 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.926597    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.927135 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.927112    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.928931 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.928911    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:23.982816 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:23.982793    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99lx6\" (UniqueName: \"kubernetes.io/projected/e5c71e6e-8806-4884-8329-987e224d23b2-kube-api-access-99lx6\") pod \"maas-api-revoke-sub-79805ca9-hqmfk\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:24.108451 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:24.108404    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:24.486378 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:24.486339    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"]
Oct 30 22:00:24.488697 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:00:24.488663    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c71e6e_8806_4884_8329_987e224d23b2.slice/crio-f2541fb0babaa9ff53bba51449c4834beb8f7ba2d21458e24b5baaabca2a3503 WatchSource:0}: Error finding container f2541fb0babaa9ff53bba51449c4834beb8f7ba2d21458e24b5baaabca2a3503: Status 404 returned error can't find the container with id f2541fb0babaa9ff53bba51449c4834beb8f7ba2d21458e24b5baaabca2a3503
Oct 30 22:00:25.444680 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:25.444653    5786 generic.go:356] "Generic (PLEG): container finished" podID="e5c71e6e-8806-4884-8329-987e224d23b2" containerID="535a68b42c07490a34408e5e7425979e276637a78a723fea761fc53c9fec0362" exitCode=0
Oct 30 22:00:25.445037 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:25.444740    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk" event={"ID":"e5c71e6e-8806-4884-8329-987e224d23b2","Type":"ContainerDied","Data":"535a68b42c07490a34408e5e7425979e276637a78a723fea761fc53c9fec0362"}
Oct 30 22:00:25.445037 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:25.444799    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk" event={"ID":"e5c71e6e-8806-4884-8329-987e224d23b2","Type":"ContainerStarted","Data":"f2541fb0babaa9ff53bba51449c4834beb8f7ba2d21458e24b5baaabca2a3503"}
Oct 30 22:00:26.583620 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.583599    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:26.650899 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.650875    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-xks-ca\") pod \"e5c71e6e-8806-4884-8329-987e224d23b2\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") "
Oct 30 22:00:26.651011 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.650982    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-openshift-ca\") pod \"e5c71e6e-8806-4884-8329-987e224d23b2\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") "
Oct 30 22:00:26.651074 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.651024    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5c71e6e-8806-4884-8329-987e224d23b2-kube-api-access-99lx6\" (UniqueName: \"kubernetes.io/projected/e5c71e6e-8806-4884-8329-987e224d23b2-kube-api-access-99lx6\") pod \"e5c71e6e-8806-4884-8329-987e224d23b2\" (UID: \"e5c71e6e-8806-4884-8329-987e224d23b2\") "
Oct 30 22:00:26.651374 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.651351    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-openshift-ca" pod "e5c71e6e-8806-4884-8329-987e224d23b2" (UID: "e5c71e6e-8806-4884-8329-987e224d23b2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:26.652974 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.652955    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-xks-ca" pod "e5c71e6e-8806-4884-8329-987e224d23b2" (UID: "e5c71e6e-8806-4884-8329-987e224d23b2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:26.653165 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.653147    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c71e6e-8806-4884-8329-987e224d23b2-kube-api-access-99lx6" pod "e5c71e6e-8806-4884-8329-987e224d23b2" (UID: "e5c71e6e-8806-4884-8329-987e224d23b2"). InnerVolumeSpecName "kube-api-access-99lx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:26.751870 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.751798    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:26.751870 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.751824    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99lx6\" (UniqueName: \"kubernetes.io/projected/e5c71e6e-8806-4884-8329-987e224d23b2-kube-api-access-99lx6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:26.751870 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:26.751835    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5c71e6e-8806-4884-8329-987e224d23b2-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:27.455978 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:27.455941    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk" event={"ID":"e5c71e6e-8806-4884-8329-987e224d23b2","Type":"ContainerDied","Data":"f2541fb0babaa9ff53bba51449c4834beb8f7ba2d21458e24b5baaabca2a3503"}
Oct 30 22:00:27.455978 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:27.455963    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"
Oct 30 22:00:27.456200 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:27.455991    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2541fb0babaa9ff53bba51449c4834beb8f7ba2d21458e24b5baaabca2a3503"
Oct 30 22:00:27.621041 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:27.621015    5786 scope.go:122] "RemoveContainer" containerID="5d756a57f2c0aa964675eaed7a03aca84218403b7b0b63ab2d900fc27a0de4e8"
Oct 30 22:00:52.354079 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:00:52.354005    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:00:54.050382 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:54.050291    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"]
Oct 30 22:00:54.099667 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:54.099637    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-fsc5x"]
Oct 30 22:00:54.359142 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:00:54.359074    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a668a3e8-c9b1-455e-bf42-973b1c7f3cc3" path="/var/lib/kubelet/pods/a668a3e8-c9b1-455e-bf42-973b1c7f3cc3/volumes"
Oct 30 22:01:04.458150 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.458117    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"]
Oct 30 22:01:04.458625 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.458545    5786 cpu_manager.go:414] "containerMap: removing container" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:01:04.458625 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.458556    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:01:04.458625 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.458571    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e5c71e6e-8806-4884-8329-987e224d23b2" containerName="revoke-keys"
Oct 30 22:01:04.458625 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.458576    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5c71e6e-8806-4884-8329-987e224d23b2" containerName="revoke-keys"
Oct 30 22:01:04.458764 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.458661    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="82acf44a-fdee-485d-a483-42749465d267" containerName="cleanup"
Oct 30 22:01:04.458764 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.458671    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5c71e6e-8806-4884-8329-987e224d23b2" containerName="revoke-keys"
Oct 30 22:01:04.462024 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.462009    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.513767 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.513742    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:01:04.513889 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.513854    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:01:04.545741 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.545716    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:01:04.562966 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.562937    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:01:04.564884 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.564867    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"]
Oct 30 22:01:04.591584 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.591562    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.591671 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.591589    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.591671 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.591630    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdv8d\" (UniqueName: \"kubernetes.io/projected/11007f5d-6d51-4243-aa19-ca15195cdf6a-kube-api-access-cdv8d\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.693185 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.693161    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.693312 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.693189    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.693312 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.693242    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdv8d\" (UniqueName: \"kubernetes.io/projected/11007f5d-6d51-4243-aa19-ca15195cdf6a-kube-api-access-cdv8d\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.693800 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.693775    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.695593 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.695574    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.749918 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.749869    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdv8d\" (UniqueName: \"kubernetes.io/projected/11007f5d-6d51-4243-aa19-ca15195cdf6a-kube-api-access-cdv8d\") pod \"maas-api-revoke-sub-0b7e8edb-zn5nm\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:04.774453 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:04.774435    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:05.145713 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:05.145691    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"]
Oct 30 22:01:05.148008 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:01:05.147979    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11007f5d_6d51_4243_aa19_ca15195cdf6a.slice/crio-2e46fec78936105b8819abb33c08e4f7186a57114bbbb916a77f7231cf3e9445 WatchSource:0}: Error finding container 2e46fec78936105b8819abb33c08e4f7186a57114bbbb916a77f7231cf3e9445: Status 404 returned error can't find the container with id 2e46fec78936105b8819abb33c08e4f7186a57114bbbb916a77f7231cf3e9445
Oct 30 22:01:05.655086 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:05.655061    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm" event={"ID":"11007f5d-6d51-4243-aa19-ca15195cdf6a","Type":"ContainerStarted","Data":"b28dc1cc74cafe6b846a1dc63d1d9c32642937ef44a5bec1173ef6ef1e7d4b8f"}
Oct 30 22:01:05.655417 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:05.655107    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm" event={"ID":"11007f5d-6d51-4243-aa19-ca15195cdf6a","Type":"ContainerStarted","Data":"2e46fec78936105b8819abb33c08e4f7186a57114bbbb916a77f7231cf3e9445"}
Oct 30 22:01:05.770912 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:05.770838    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm" podStartSLOduration=1.770825538 podStartE2EDuration="1.770825538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:05.768965792 +0000 UTC m=+2921.640084911" watchObservedRunningTime="2026-10-30 22:01:05.770825538 +0000 UTC m=+2921.641944691"
Oct 30 22:01:06.661859 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:06.661824    5786 generic.go:356] "Generic (PLEG): container finished" podID="11007f5d-6d51-4243-aa19-ca15195cdf6a" containerID="b28dc1cc74cafe6b846a1dc63d1d9c32642937ef44a5bec1173ef6ef1e7d4b8f" exitCode=0
Oct 30 22:01:06.662287 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:06.661909    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm" event={"ID":"11007f5d-6d51-4243-aa19-ca15195cdf6a","Type":"ContainerDied","Data":"b28dc1cc74cafe6b846a1dc63d1d9c32642937ef44a5bec1173ef6ef1e7d4b8f"}
Oct 30 22:01:07.798343 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:07.798322    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:07.920648 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:07.920576    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-openshift-ca\") pod \"11007f5d-6d51-4243-aa19-ca15195cdf6a\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") "
Oct 30 22:01:07.920648 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:07.920624    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-xks-ca\") pod \"11007f5d-6d51-4243-aa19-ca15195cdf6a\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") "
Oct 30 22:01:07.920870 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:07.920655    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11007f5d-6d51-4243-aa19-ca15195cdf6a-kube-api-access-cdv8d\" (UniqueName: \"kubernetes.io/projected/11007f5d-6d51-4243-aa19-ca15195cdf6a-kube-api-access-cdv8d\") pod \"11007f5d-6d51-4243-aa19-ca15195cdf6a\" (UID: \"11007f5d-6d51-4243-aa19-ca15195cdf6a\") "
Oct 30 22:01:07.920973 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:07.920947    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-openshift-ca" pod "11007f5d-6d51-4243-aa19-ca15195cdf6a" (UID: "11007f5d-6d51-4243-aa19-ca15195cdf6a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:07.921109 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:07.921091    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:07.922769 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:07.922742    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-xks-ca" pod "11007f5d-6d51-4243-aa19-ca15195cdf6a" (UID: "11007f5d-6d51-4243-aa19-ca15195cdf6a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:01:07.922939 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:07.922918    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11007f5d-6d51-4243-aa19-ca15195cdf6a-kube-api-access-cdv8d" pod "11007f5d-6d51-4243-aa19-ca15195cdf6a" (UID: "11007f5d-6d51-4243-aa19-ca15195cdf6a"). InnerVolumeSpecName "kube-api-access-cdv8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:08.022477 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:08.022453    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11007f5d-6d51-4243-aa19-ca15195cdf6a-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:08.022477 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:08.022475    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdv8d\" (UniqueName: \"kubernetes.io/projected/11007f5d-6d51-4243-aa19-ca15195cdf6a-kube-api-access-cdv8d\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:08.673539 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:08.673508    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"
Oct 30 22:01:08.673706 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:08.673542    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm" event={"ID":"11007f5d-6d51-4243-aa19-ca15195cdf6a","Type":"ContainerDied","Data":"2e46fec78936105b8819abb33c08e4f7186a57114bbbb916a77f7231cf3e9445"}
Oct 30 22:01:08.673706 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:08.673582    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e46fec78936105b8819abb33c08e4f7186a57114bbbb916a77f7231cf3e9445"
Oct 30 22:01:15.877022 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.876987    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"]
Oct 30 22:01:15.877441 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.877427    5786 cpu_manager.go:414] "containerMap: removing container" podUID="11007f5d-6d51-4243-aa19-ca15195cdf6a" containerName="revoke-keys"
Oct 30 22:01:15.877486 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.877442    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11007f5d-6d51-4243-aa19-ca15195cdf6a" containerName="revoke-keys"
Oct 30 22:01:15.877556 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.877546    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="11007f5d-6d51-4243-aa19-ca15195cdf6a" containerName="revoke-keys"
Oct 30 22:01:15.880379 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.880362    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:15.887112 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.886768    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcmwf\" (UniqueName: \"kubernetes.io/projected/389ef97f-9b8f-4541-b4cb-568f5438ebf2-kube-api-access-lcmwf\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:15.887112 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.886841    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:15.887112 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.887010    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:15.932574 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.932544    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:01:15.932722 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.932544    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:01:15.932722 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.932637    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:01:15.981927 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.981903    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:01:15.983850 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.983831    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"]
Oct 30 22:01:15.987666 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.987633    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:15.987744 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.987700    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcmwf\" (UniqueName: \"kubernetes.io/projected/389ef97f-9b8f-4541-b4cb-568f5438ebf2-kube-api-access-lcmwf\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:15.987791 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.987745    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:15.988258 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.988239    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:15.990167 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:15.990146    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:16.045346 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:16.045320    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcmwf\" (UniqueName: \"kubernetes.io/projected/389ef97f-9b8f-4541-b4cb-568f5438ebf2-kube-api-access-lcmwf\") pod \"maas-api-revoke-sub-3d4156f1-qt5c4\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:16.193498 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:16.193473    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:16.567623 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:16.567568    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"]
Oct 30 22:01:16.570118 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:01:16.570091    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389ef97f_9b8f_4541_b4cb_568f5438ebf2.slice/crio-a0f2a21e287b18622f04653f4d8e4242ed9b22e560e53056453df6cb9cfa5126 WatchSource:0}: Error finding container a0f2a21e287b18622f04653f4d8e4242ed9b22e560e53056453df6cb9cfa5126: Status 404 returned error can't find the container with id a0f2a21e287b18622f04653f4d8e4242ed9b22e560e53056453df6cb9cfa5126
Oct 30 22:01:16.714730 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:16.714699    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4" event={"ID":"389ef97f-9b8f-4541-b4cb-568f5438ebf2","Type":"ContainerStarted","Data":"a0f2a21e287b18622f04653f4d8e4242ed9b22e560e53056453df6cb9cfa5126"}
Oct 30 22:01:17.723907 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:17.723871    5786 generic.go:356] "Generic (PLEG): container finished" podID="389ef97f-9b8f-4541-b4cb-568f5438ebf2" containerID="b0efb7738705eb31abd89fc61eba1d3bf7263a2c149184dcaf999ef528a47ce7" exitCode=0
Oct 30 22:01:17.724438 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:17.723950    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4" event={"ID":"389ef97f-9b8f-4541-b4cb-568f5438ebf2","Type":"ContainerDied","Data":"b0efb7738705eb31abd89fc61eba1d3bf7263a2c149184dcaf999ef528a47ce7"}
Oct 30 22:01:19.496564 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.484144    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:19.619433 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.619391    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-openshift-ca\") pod \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") "
Oct 30 22:01:19.619638 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.619472    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-xks-ca\") pod \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") "
Oct 30 22:01:19.619638 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.619495    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/389ef97f-9b8f-4541-b4cb-568f5438ebf2-kube-api-access-lcmwf\" (UniqueName: \"kubernetes.io/projected/389ef97f-9b8f-4541-b4cb-568f5438ebf2-kube-api-access-lcmwf\") pod \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\" (UID: \"389ef97f-9b8f-4541-b4cb-568f5438ebf2\") "
Oct 30 22:01:19.620170 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.620142    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-openshift-ca" pod "389ef97f-9b8f-4541-b4cb-568f5438ebf2" (UID: "389ef97f-9b8f-4541-b4cb-568f5438ebf2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:19.622148 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.622092    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/389ef97f-9b8f-4541-b4cb-568f5438ebf2-kube-api-access-lcmwf" pod "389ef97f-9b8f-4541-b4cb-568f5438ebf2" (UID: "389ef97f-9b8f-4541-b4cb-568f5438ebf2"). InnerVolumeSpecName "kube-api-access-lcmwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:19.622390 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.622360    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-xks-ca" pod "389ef97f-9b8f-4541-b4cb-568f5438ebf2" (UID: "389ef97f-9b8f-4541-b4cb-568f5438ebf2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:01:19.721150 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.721125    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:19.721150 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.721149    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcmwf\" (UniqueName: \"kubernetes.io/projected/389ef97f-9b8f-4541-b4cb-568f5438ebf2-kube-api-access-lcmwf\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:19.721330 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.721159    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389ef97f-9b8f-4541-b4cb-568f5438ebf2-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:19.746119 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.746094    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"
Oct 30 22:01:19.746266 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.746094    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4" event={"ID":"389ef97f-9b8f-4541-b4cb-568f5438ebf2","Type":"ContainerDied","Data":"a0f2a21e287b18622f04653f4d8e4242ed9b22e560e53056453df6cb9cfa5126"}
Oct 30 22:01:19.746326 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:19.746266    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0f2a21e287b18622f04653f4d8e4242ed9b22e560e53056453df6cb9cfa5126"
Oct 30 22:01:22.353409 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:01:22.353373    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:01:27.645292 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:27.645259    5786 scope.go:122] "RemoveContainer" containerID="2d0403e4cdffbff023c82d6b0fe758e7c03331c4e1b6b0db9c59b4cc301ede3f"
Oct 30 22:01:27.653967 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:27.653946    5786 scope.go:122] "RemoveContainer" containerID="f3eea40a14904b816d8a6e78c6505f8d21b4039c45bfff0da289b59c00d221c6"
Oct 30 22:01:46.429096 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.429064    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"]
Oct 30 22:01:46.478262 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.478233    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-qt5c4"]
Oct 30 22:01:46.539269 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.539230    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"]
Oct 30 22:01:46.539716 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.539699    5786 cpu_manager.go:414] "containerMap: removing container" podUID="389ef97f-9b8f-4541-b4cb-568f5438ebf2" containerName="revoke-keys"
Oct 30 22:01:46.539831 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.539718    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389ef97f-9b8f-4541-b4cb-568f5438ebf2" containerName="revoke-keys"
Oct 30 22:01:46.539831 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.539819    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="389ef97f-9b8f-4541-b4cb-568f5438ebf2" containerName="revoke-keys"
Oct 30 22:01:46.544217 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.544187    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.597632 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.597480    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:01:46.597632 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.597594    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:01:46.598355 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.597709    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:01:46.598355 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.597850    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:01:46.599693 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.599674    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"]
Oct 30 22:01:46.601941 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.601920    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.602043 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.601955    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s289k\" (UniqueName: \"kubernetes.io/projected/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-kube-api-access-s289k\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.602109 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.602089    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.702984 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.702926    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.703094 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.702996    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.703094 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.703020    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s289k\" (UniqueName: \"kubernetes.io/projected/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-kube-api-access-s289k\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.703610 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.703582    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.705457 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.705441    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.759095 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.759069    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s289k\" (UniqueName: \"kubernetes.io/projected/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-kube-api-access-s289k\") pod \"maas-api-revoke-sub-3d4156f1-wngqc\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:46.856300 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:46.856280    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:47.229855 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:47.229809    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"]
Oct 30 22:01:47.232197 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:01:47.232172    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95a028e_42f7_4b9c_85d7_14c68ed03d6f.slice/crio-7eeabdf4ce8a9bd216f48aeb2f04866b615c6569f9b5e2cb5d64ef5cad2ccc58 WatchSource:0}: Error finding container 7eeabdf4ce8a9bd216f48aeb2f04866b615c6569f9b5e2cb5d64ef5cad2ccc58: Status 404 returned error can't find the container with id 7eeabdf4ce8a9bd216f48aeb2f04866b615c6569f9b5e2cb5d64ef5cad2ccc58
Oct 30 22:01:47.946350 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:47.946318    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc" event={"ID":"f95a028e-42f7-4b9c-85d7-14c68ed03d6f","Type":"ContainerStarted","Data":"6a2daefc5e1d119644f7b1ed5dd6e3693b3db89ae5bc6a3db51140c4b4df5078"}
Oct 30 22:01:47.946694 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:47.946372    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc" event={"ID":"f95a028e-42f7-4b9c-85d7-14c68ed03d6f","Type":"ContainerStarted","Data":"7eeabdf4ce8a9bd216f48aeb2f04866b615c6569f9b5e2cb5d64ef5cad2ccc58"}
Oct 30 22:01:48.062454 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:48.062411    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc" podStartSLOduration=2.062397168 podStartE2EDuration="2.062397168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:48.060784733 +0000 UTC m=+2963.931903855" watchObservedRunningTime="2026-10-30 22:01:48.062397168 +0000 UTC m=+2963.933516287"
Oct 30 22:01:48.357432 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:48.357373    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="389ef97f-9b8f-4541-b4cb-568f5438ebf2" path="/var/lib/kubelet/pods/389ef97f-9b8f-4541-b4cb-568f5438ebf2/volumes"
Oct 30 22:01:48.953165 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:48.953140    5786 generic.go:356] "Generic (PLEG): container finished" podID="f95a028e-42f7-4b9c-85d7-14c68ed03d6f" containerID="6a2daefc5e1d119644f7b1ed5dd6e3693b3db89ae5bc6a3db51140c4b4df5078" exitCode=0
Oct 30 22:01:48.953539 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:48.953187    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc" event={"ID":"f95a028e-42f7-4b9c-85d7-14c68ed03d6f","Type":"ContainerDied","Data":"6a2daefc5e1d119644f7b1ed5dd6e3693b3db89ae5bc6a3db51140c4b4df5078"}
Oct 30 22:01:50.090633 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.090612    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:50.133738 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.133713    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-xks-ca\") pod \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") "
Oct 30 22:01:50.133845 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.133763    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-openshift-ca\") pod \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") "
Oct 30 22:01:50.133845 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.133808    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-kube-api-access-s289k\" (UniqueName: \"kubernetes.io/projected/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-kube-api-access-s289k\") pod \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\" (UID: \"f95a028e-42f7-4b9c-85d7-14c68ed03d6f\") "
Oct 30 22:01:50.134097 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.134075    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-openshift-ca" pod "f95a028e-42f7-4b9c-85d7-14c68ed03d6f" (UID: "f95a028e-42f7-4b9c-85d7-14c68ed03d6f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:50.134243 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.134199    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:50.135801 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.135777    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-xks-ca" pod "f95a028e-42f7-4b9c-85d7-14c68ed03d6f" (UID: "f95a028e-42f7-4b9c-85d7-14c68ed03d6f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:01:50.135979 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.135960    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-kube-api-access-s289k" pod "f95a028e-42f7-4b9c-85d7-14c68ed03d6f" (UID: "f95a028e-42f7-4b9c-85d7-14c68ed03d6f"). InnerVolumeSpecName "kube-api-access-s289k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:50.235266 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.235192    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s289k\" (UniqueName: \"kubernetes.io/projected/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-kube-api-access-s289k\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:50.235266 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.235237    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f95a028e-42f7-4b9c-85d7-14c68ed03d6f-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:01:50.964038 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.964002    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc" event={"ID":"f95a028e-42f7-4b9c-85d7-14c68ed03d6f","Type":"ContainerDied","Data":"7eeabdf4ce8a9bd216f48aeb2f04866b615c6569f9b5e2cb5d64ef5cad2ccc58"}
Oct 30 22:01:50.964201 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.964077    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eeabdf4ce8a9bd216f48aeb2f04866b615c6569f9b5e2cb5d64ef5cad2ccc58"
Oct 30 22:01:50.964201 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:01:50.964021    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"
Oct 30 22:01:52.354047 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:01:52.354017    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:02:04.052270 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:04.052172    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"]
Oct 30 22:02:04.101377 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:04.101347    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-rbbg8"]
Oct 30 22:02:04.357657 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:04.357581    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="016aeeb8-a2f8-4c34-9956-f2dbc02b76e8" path="/var/lib/kubelet/pods/016aeeb8-a2f8-4c34-9956-f2dbc02b76e8/volumes"
Oct 30 22:02:22.354521 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:02:22.354487    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:02:27.685980 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:27.685298    5786 scope.go:122] "RemoveContainer" containerID="37a6ff76a8cc9f254e9053de5f5bee19a049063fd9b38304b925cbb37e95caaf"
Oct 30 22:02:28.589482 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:28.589437    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-zr2km container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:02:28.589873 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:28.589824    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:02:32.047546 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:32.047504    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"]
Oct 30 22:02:32.096948 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:32.096907    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5vhk9"]
Oct 30 22:02:32.359101 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:32.359016    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f03f26b0-9253-42d4-b92d-f2254b6aba10" path="/var/lib/kubelet/pods/f03f26b0-9253-42d4-b92d-f2254b6aba10/volumes"
Oct 30 22:02:45.061186 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:45.061142    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"]
Oct 30 22:02:45.110632 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:45.110565    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-t5jfl"]
Oct 30 22:02:46.006177 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.006131    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"]
Oct 30 22:02:46.006625 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.006610    5786 cpu_manager.go:414] "containerMap: removing container" podUID="f95a028e-42f7-4b9c-85d7-14c68ed03d6f" containerName="revoke-keys"
Oct 30 22:02:46.006676 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.006626    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f95a028e-42f7-4b9c-85d7-14c68ed03d6f" containerName="revoke-keys"
Oct 30 22:02:46.006713 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.006696    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="f95a028e-42f7-4b9c-85d7-14c68ed03d6f" containerName="revoke-keys"
Oct 30 22:02:46.011199 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.011180    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.067162 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.067126    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"]
Oct 30 22:02:46.112957 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.112922    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n826z\" (UniqueName: \"kubernetes.io/projected/8acec2ee-7410-4c77-9ed1-d3495da3ece0-kube-api-access-n826z\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.113165 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.113056    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.113165 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.113137    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.113810 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.113776    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:02:46.113810 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.113798    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:02:46.113966 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.113828    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:02:46.113966 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.113902    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:02:46.214197 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.214159    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.214430 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.214251    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n826z\" (UniqueName: \"kubernetes.io/projected/8acec2ee-7410-4c77-9ed1-d3495da3ece0-kube-api-access-n826z\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.214430 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.214347    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.215015 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.214990    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.216725 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.216695    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.270748 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.270663    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n826z\" (UniqueName: \"kubernetes.io/projected/8acec2ee-7410-4c77-9ed1-d3495da3ece0-kube-api-access-n826z\") pod \"maas-api-revoke-sub-e2fc9c5b-kqhhl\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.324467 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.324432    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:46.358932 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.358901    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="146cc47e-d072-49ca-acd6-8d85fa686f7c" path="/var/lib/kubelet/pods/146cc47e-d072-49ca-acd6-8d85fa686f7c/volumes"
Oct 30 22:02:46.704813 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:46.704783    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"]
Oct 30 22:02:46.707722 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:02:46.707686    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8acec2ee_7410_4c77_9ed1_d3495da3ece0.slice/crio-1c4b3c0286bb89572949db695ef1d3ba4424606e42614d7311409ed9f3e65509 WatchSource:0}: Error finding container 1c4b3c0286bb89572949db695ef1d3ba4424606e42614d7311409ed9f3e65509: Status 404 returned error can't find the container with id 1c4b3c0286bb89572949db695ef1d3ba4424606e42614d7311409ed9f3e65509
Oct 30 22:02:47.295589 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:47.295511    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl" event={"ID":"8acec2ee-7410-4c77-9ed1-d3495da3ece0","Type":"ContainerStarted","Data":"88c997265f0767e1344aba3a366bae335e259f56abdef54018b40233804fb822"}
Oct 30 22:02:47.295589 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:47.295565    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl" event={"ID":"8acec2ee-7410-4c77-9ed1-d3495da3ece0","Type":"ContainerStarted","Data":"1c4b3c0286bb89572949db695ef1d3ba4424606e42614d7311409ed9f3e65509"}
Oct 30 22:02:47.413561 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:47.413509    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl" podStartSLOduration=2.413483702 podStartE2EDuration="2.413483702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:47.412652902 +0000 UTC m=+3023.283772023" watchObservedRunningTime="2026-10-30 22:02:47.413483702 +0000 UTC m=+3023.284602824"
Oct 30 22:02:48.303333 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:48.303299    5786 generic.go:356] "Generic (PLEG): container finished" podID="8acec2ee-7410-4c77-9ed1-d3495da3ece0" containerID="88c997265f0767e1344aba3a366bae335e259f56abdef54018b40233804fb822" exitCode=0
Oct 30 22:02:48.303747 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:48.303350    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl" event={"ID":"8acec2ee-7410-4c77-9ed1-d3495da3ece0","Type":"ContainerDied","Data":"88c997265f0767e1344aba3a366bae335e259f56abdef54018b40233804fb822"}
Oct 30 22:02:49.452964 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.452934    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:49.548576 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.548537    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8acec2ee-7410-4c77-9ed1-d3495da3ece0-kube-api-access-n826z\" (UniqueName: \"kubernetes.io/projected/8acec2ee-7410-4c77-9ed1-d3495da3ece0-kube-api-access-n826z\") pod \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") "
Oct 30 22:02:49.548795 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.548641    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-openshift-ca\") pod \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") "
Oct 30 22:02:49.548795 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.548731    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-xks-ca\") pod \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\" (UID: \"8acec2ee-7410-4c77-9ed1-d3495da3ece0\") "
Oct 30 22:02:49.549000 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.548976    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-openshift-ca" pod "8acec2ee-7410-4c77-9ed1-d3495da3ece0" (UID: "8acec2ee-7410-4c77-9ed1-d3495da3ece0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:49.558471 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.558414    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-xks-ca" pod "8acec2ee-7410-4c77-9ed1-d3495da3ece0" (UID: "8acec2ee-7410-4c77-9ed1-d3495da3ece0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:49.559789 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.559721    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8acec2ee-7410-4c77-9ed1-d3495da3ece0-kube-api-access-n826z" pod "8acec2ee-7410-4c77-9ed1-d3495da3ece0" (UID: "8acec2ee-7410-4c77-9ed1-d3495da3ece0"). InnerVolumeSpecName "kube-api-access-n826z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:49.651003 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.650961    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:49.651191 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.651018    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n826z\" (UniqueName: \"kubernetes.io/projected/8acec2ee-7410-4c77-9ed1-d3495da3ece0-kube-api-access-n826z\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:49.651191 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:49.651030    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8acec2ee-7410-4c77-9ed1-d3495da3ece0-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.315559 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:50.315528    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"
Oct 30 22:02:50.315752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:50.315559    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl" event={"ID":"8acec2ee-7410-4c77-9ed1-d3495da3ece0","Type":"ContainerDied","Data":"1c4b3c0286bb89572949db695ef1d3ba4424606e42614d7311409ed9f3e65509"}
Oct 30 22:02:50.315752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:50.315635    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c4b3c0286bb89572949db695ef1d3ba4424606e42614d7311409ed9f3e65509"
Oct 30 22:02:52.357720 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:02:52.357671    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:02:56.776422 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.776382    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"]
Oct 30 22:02:56.776930 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.776912    5786 cpu_manager.go:414] "containerMap: removing container" podUID="8acec2ee-7410-4c77-9ed1-d3495da3ece0" containerName="revoke-keys"
Oct 30 22:02:56.776978 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.776935    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8acec2ee-7410-4c77-9ed1-d3495da3ece0" containerName="revoke-keys"
Oct 30 22:02:56.777084 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.777072    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="8acec2ee-7410-4c77-9ed1-d3495da3ece0" containerName="revoke-keys"
Oct 30 22:02:56.781229 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.781177    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:56.884034 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.884001    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:02:56.884234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.884108    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:02:56.884300 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.884287    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:02:56.884347 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.884336    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:02:56.886098 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.886080    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"]
Oct 30 22:02:56.928673 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.928642    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:56.928864 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.928701    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nc74\" (UniqueName: \"kubernetes.io/projected/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-kube-api-access-5nc74\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:56.928864 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:56.928821    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:57.030031 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:57.029943    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:57.030031 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:57.030025    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:57.030266 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:57.030073    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nc74\" (UniqueName: \"kubernetes.io/projected/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-kube-api-access-5nc74\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:57.030740 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:57.030722    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:57.032637 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:57.032615    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:57.087850 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:57.087814    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nc74\" (UniqueName: \"kubernetes.io/projected/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-kube-api-access-5nc74\") pod \"maas-api-revoke-sub-5a29efb3-h4zm4\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:57.094086 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:57.094061    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:57.467158 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:57.467131    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"]
Oct 30 22:02:57.470084 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:02:57.470056    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737e56e0_5dd6_4965_9ee0_0a6ed20054d5.slice/crio-293e4d04486c5481f1d208dc3bbf40732b4e745ad2c4e53116e6b3f6c43552a4 WatchSource:0}: Error finding container 293e4d04486c5481f1d208dc3bbf40732b4e745ad2c4e53116e6b3f6c43552a4: Status 404 returned error can't find the container with id 293e4d04486c5481f1d208dc3bbf40732b4e745ad2c4e53116e6b3f6c43552a4
Oct 30 22:02:58.363595 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:58.363564    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4" event={"ID":"737e56e0-5dd6-4965-9ee0-0a6ed20054d5","Type":"ContainerStarted","Data":"0e6caf010fe17a6bd222cf0831b1289f4595bf40f246f0fd2f268f43bbde5980"}
Oct 30 22:02:58.363935 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:58.363617    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4" event={"ID":"737e56e0-5dd6-4965-9ee0-0a6ed20054d5","Type":"ContainerStarted","Data":"293e4d04486c5481f1d208dc3bbf40732b4e745ad2c4e53116e6b3f6c43552a4"}
Oct 30 22:02:59.374617 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.374574    5786 generic.go:356] "Generic (PLEG): container finished" podID="737e56e0-5dd6-4965-9ee0-0a6ed20054d5" containerID="0e6caf010fe17a6bd222cf0831b1289f4595bf40f246f0fd2f268f43bbde5980" exitCode=0
Oct 30 22:02:59.375174 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.374643    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4" event={"ID":"737e56e0-5dd6-4965-9ee0-0a6ed20054d5","Type":"ContainerDied","Data":"0e6caf010fe17a6bd222cf0831b1289f4595bf40f246f0fd2f268f43bbde5980"}
Oct 30 22:02:59.570758 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.570720    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:02:59.666709 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.666663    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-xks-ca\") pod \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") "
Oct 30 22:02:59.666931 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.666796    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-openshift-ca\") pod \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") "
Oct 30 22:02:59.666931 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.666835    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-kube-api-access-5nc74\" (UniqueName: \"kubernetes.io/projected/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-kube-api-access-5nc74\") pod \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\" (UID: \"737e56e0-5dd6-4965-9ee0-0a6ed20054d5\") "
Oct 30 22:02:59.667840 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.667194    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-openshift-ca" pod "737e56e0-5dd6-4965-9ee0-0a6ed20054d5" (UID: "737e56e0-5dd6-4965-9ee0-0a6ed20054d5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:59.667840 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.667364    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:59.670067 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.670017    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-xks-ca" pod "737e56e0-5dd6-4965-9ee0-0a6ed20054d5" (UID: "737e56e0-5dd6-4965-9ee0-0a6ed20054d5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:59.670067 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.670045    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-kube-api-access-5nc74" pod "737e56e0-5dd6-4965-9ee0-0a6ed20054d5" (UID: "737e56e0-5dd6-4965-9ee0-0a6ed20054d5"). InnerVolumeSpecName "kube-api-access-5nc74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:59.769078 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.769026    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:59.769078 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:02:59.769068    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nc74\" (UniqueName: \"kubernetes.io/projected/737e56e0-5dd6-4965-9ee0-0a6ed20054d5-kube-api-access-5nc74\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:00.454247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:00.450642    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4" event={"ID":"737e56e0-5dd6-4965-9ee0-0a6ed20054d5","Type":"ContainerDied","Data":"293e4d04486c5481f1d208dc3bbf40732b4e745ad2c4e53116e6b3f6c43552a4"}
Oct 30 22:03:00.454247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:00.450724    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="293e4d04486c5481f1d208dc3bbf40732b4e745ad2c4e53116e6b3f6c43552a4"
Oct 30 22:03:00.454247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:00.450823    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"
Oct 30 22:03:14.140229 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.138510    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"]
Oct 30 22:03:14.186907 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.186863    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-wngqc"]
Oct 30 22:03:14.249595 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.249558    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"]
Oct 30 22:03:14.250135 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.250115    5786 cpu_manager.go:414] "containerMap: removing container" podUID="737e56e0-5dd6-4965-9ee0-0a6ed20054d5" containerName="revoke-keys"
Oct 30 22:03:14.250279 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.250137    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="737e56e0-5dd6-4965-9ee0-0a6ed20054d5" containerName="revoke-keys"
Oct 30 22:03:14.250358 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.250315    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="737e56e0-5dd6-4965-9ee0-0a6ed20054d5" containerName="revoke-keys"
Oct 30 22:03:14.253695 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.253672    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.305748 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.305701    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:03:14.305748 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.305724    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:03:14.305748 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.305736    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:03:14.306049 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.305773    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:03:14.312884 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.312850    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"]
Oct 30 22:03:14.358455 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.358422    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f95a028e-42f7-4b9c-85d7-14c68ed03d6f" path="/var/lib/kubelet/pods/f95a028e-42f7-4b9c-85d7-14c68ed03d6f/volumes"
Oct 30 22:03:14.376740 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.376701    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.376954 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.376764    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmxms\" (UniqueName: \"kubernetes.io/projected/d474d067-bf6f-40ef-bfd6-2b2402ddda15-kube-api-access-vmxms\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.376954 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.376803    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.477978 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.477938    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.478190 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.478096    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.478190 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.478137    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmxms\" (UniqueName: \"kubernetes.io/projected/d474d067-bf6f-40ef-bfd6-2b2402ddda15-kube-api-access-vmxms\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.478792 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.478765    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.480725 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.480692    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.535506 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.535470    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmxms\" (UniqueName: \"kubernetes.io/projected/d474d067-bf6f-40ef-bfd6-2b2402ddda15-kube-api-access-vmxms\") pod \"maas-api-revoke-sub-3d4156f1-84h7b\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.567883 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.567853    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:14.945896 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:03:14.945853    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd474d067_bf6f_40ef_bfd6_2b2402ddda15.slice/crio-4d29794cf0e4d13d3f9c575affcde011a7649e3ca5fd7bec3572f9a3942f0755 WatchSource:0}: Error finding container 4d29794cf0e4d13d3f9c575affcde011a7649e3ca5fd7bec3572f9a3942f0755: Status 404 returned error can't find the container with id 4d29794cf0e4d13d3f9c575affcde011a7649e3ca5fd7bec3572f9a3942f0755
Oct 30 22:03:14.957081 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:14.957057    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"]
Oct 30 22:03:15.547540 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:15.547435    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b" event={"ID":"d474d067-bf6f-40ef-bfd6-2b2402ddda15","Type":"ContainerStarted","Data":"17b3381830171744e4f5ec26005c9465439b8cbf5cc53528693130868713c390"}
Oct 30 22:03:15.547540 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:15.547489    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b" event={"ID":"d474d067-bf6f-40ef-bfd6-2b2402ddda15","Type":"ContainerStarted","Data":"4d29794cf0e4d13d3f9c575affcde011a7649e3ca5fd7bec3572f9a3942f0755"}
Oct 30 22:03:15.665307 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:15.665249    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b" podStartSLOduration=1.6652325700000001 podStartE2EDuration="1.66523257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:15.663121505 +0000 UTC m=+3051.534240639" watchObservedRunningTime="2026-10-30 22:03:15.66523257 +0000 UTC m=+3051.536351684"
Oct 30 22:03:16.554473 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:16.554434    5786 generic.go:356] "Generic (PLEG): container finished" podID="d474d067-bf6f-40ef-bfd6-2b2402ddda15" containerID="17b3381830171744e4f5ec26005c9465439b8cbf5cc53528693130868713c390" exitCode=0
Oct 30 22:03:16.554929 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:16.554514    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b" event={"ID":"d474d067-bf6f-40ef-bfd6-2b2402ddda15","Type":"ContainerDied","Data":"17b3381830171744e4f5ec26005c9465439b8cbf5cc53528693130868713c390"}
Oct 30 22:03:17.697380 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.697349    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:17.711076 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.711040    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-openshift-ca\") pod \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") "
Oct 30 22:03:17.711262 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.711244    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-xks-ca\") pod \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") "
Oct 30 22:03:17.711313 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.711297    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d474d067-bf6f-40ef-bfd6-2b2402ddda15-kube-api-access-vmxms\" (UniqueName: \"kubernetes.io/projected/d474d067-bf6f-40ef-bfd6-2b2402ddda15-kube-api-access-vmxms\") pod \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\" (UID: \"d474d067-bf6f-40ef-bfd6-2b2402ddda15\") "
Oct 30 22:03:17.711482 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.711460    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-openshift-ca" pod "d474d067-bf6f-40ef-bfd6-2b2402ddda15" (UID: "d474d067-bf6f-40ef-bfd6-2b2402ddda15"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:17.711654 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.711642    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:17.713608 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.713578    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-xks-ca" pod "d474d067-bf6f-40ef-bfd6-2b2402ddda15" (UID: "d474d067-bf6f-40ef-bfd6-2b2402ddda15"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:17.713608 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.713593    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d474d067-bf6f-40ef-bfd6-2b2402ddda15-kube-api-access-vmxms" pod "d474d067-bf6f-40ef-bfd6-2b2402ddda15" (UID: "d474d067-bf6f-40ef-bfd6-2b2402ddda15"). InnerVolumeSpecName "kube-api-access-vmxms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:17.812642 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.812469    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d474d067-bf6f-40ef-bfd6-2b2402ddda15-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:17.812642 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:17.812522    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmxms\" (UniqueName: \"kubernetes.io/projected/d474d067-bf6f-40ef-bfd6-2b2402ddda15-kube-api-access-vmxms\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:18.565787 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:18.565755    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"
Oct 30 22:03:18.566001 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:18.565784    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b" event={"ID":"d474d067-bf6f-40ef-bfd6-2b2402ddda15","Type":"ContainerDied","Data":"4d29794cf0e4d13d3f9c575affcde011a7649e3ca5fd7bec3572f9a3942f0755"}
Oct 30 22:03:18.566001 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:18.565838    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d29794cf0e4d13d3f9c575affcde011a7649e3ca5fd7bec3572f9a3942f0755"
Oct 30 22:03:22.354520 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:03:22.354488    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:03:27.824675 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:27.824641    5786 scope.go:122] "RemoveContainer" containerID="dcd9133f968a73b3708645e4c850e79df1b553484a07f1abdb0c54576efe55f2"
Oct 30 22:03:30.058325 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:30.052862    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"]
Oct 30 22:03:30.123863 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:30.123829    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-qkpqz"]
Oct 30 22:03:30.402800 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:30.402715    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff7a0fad-5eb9-4576-b480-566649a67864" path="/var/lib/kubelet/pods/ff7a0fad-5eb9-4576-b480-566649a67864/volumes"
Oct 30 22:03:41.049816 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:41.049773    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"]
Oct 30 22:03:41.099032 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:41.098994    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-4hl4k"]
Oct 30 22:03:42.358945 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:42.358913    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7dbbc227-5016-4fb3-a5c4-f476cc7ca626" path="/var/lib/kubelet/pods/7dbbc227-5016-4fb3-a5c4-f476cc7ca626/volumes"
Oct 30 22:03:46.753750 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.753714    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"]
Oct 30 22:03:46.754263 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.754137    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d474d067-bf6f-40ef-bfd6-2b2402ddda15" containerName="revoke-keys"
Oct 30 22:03:46.754263 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.754149    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d474d067-bf6f-40ef-bfd6-2b2402ddda15" containerName="revoke-keys"
Oct 30 22:03:46.754263 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.754257    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d474d067-bf6f-40ef-bfd6-2b2402ddda15" containerName="revoke-keys"
Oct 30 22:03:46.758965 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.758944    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.810879 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.810843    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-197a40-openshift-default-dockercfg-z59sv\""
Oct 30 22:03:46.852725 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.852689    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-socket\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.852725 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.852728    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zn5j\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-kube-api-access-8zn5j\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.852945 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.852811    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673ad515-c52c-48a5-8d24-b089ce633478-istio-podinfo\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.852945 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.852832    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-certs\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.852945 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.852847    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-data\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.852945 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.852868    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-envoy\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.853077 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.853005    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-istio-token\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.853077 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.853041    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673ad515-c52c-48a5-8d24-b089ce633478-istiod-ca-cert\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.853145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.853086    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-credential-socket\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.862697 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.862651    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"]
Oct 30 22:03:46.953747 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.953711    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-istio-token\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.953747 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.953751    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673ad515-c52c-48a5-8d24-b089ce633478-istiod-ca-cert\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954041 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.953935    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-credential-socket\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954116 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954055    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-socket\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954116 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954096    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zn5j\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-kube-api-access-8zn5j\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954261 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954187    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673ad515-c52c-48a5-8d24-b089ce633478-istio-podinfo\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954261 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954244    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-certs\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954444 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954269    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-data\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954444 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954311    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-envoy\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954444 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954376    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-credential-socket\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954444 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954407    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-socket\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954684 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954544    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673ad515-c52c-48a5-8d24-b089ce633478-istiod-ca-cert\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954684 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954620    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-data\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.954684 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.954623    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-certs\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.956936 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.956911    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-envoy\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:46.957316 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:46.957293    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673ad515-c52c-48a5-8d24-b089ce633478-istio-podinfo\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:47.010626 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.010551    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-istio-token\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:47.010896 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.010871    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zn5j\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-kube-api-access-8zn5j\") pod \"e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") " pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:47.086700 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.086650    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:47.521077 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.521047    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"]
Oct 30 22:03:47.523407 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:03:47.523378    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673ad515_c52c_48a5_8d24_b089ce633478.slice/crio-764a030038194d0b13bdf82043219af3c5385aa2654a78cc52ff5ed67d95204a WatchSource:0}: Error finding container 764a030038194d0b13bdf82043219af3c5385aa2654a78cc52ff5ed67d95204a: Status 404 returned error can't find the container with id 764a030038194d0b13bdf82043219af3c5385aa2654a78cc52ff5ed67d95204a
Oct 30 22:03:47.525693 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.525662    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:03:47.525769 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.525729    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:03:47.525769 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.525756    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:03:47.760579 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.760536    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" event={"ID":"673ad515-c52c-48a5-8d24-b089ce633478","Type":"ContainerStarted","Data":"79356bf06ca1f1d2ab64338681839b5fe35081eaf1f549e266c0193655ea5224"}
Oct 30 22:03:47.761049 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.760612    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" event={"ID":"673ad515-c52c-48a5-8d24-b089ce633478","Type":"ContainerStarted","Data":"764a030038194d0b13bdf82043219af3c5385aa2654a78cc52ff5ed67d95204a"}
Oct 30 22:03:47.880090 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:47.879980    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" podStartSLOduration=1.879961287 podStartE2EDuration="1.879961287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:47.877856405 +0000 UTC m=+3083.748975527" watchObservedRunningTime="2026-10-30 22:03:47.879961287 +0000 UTC m=+3083.751080406"
Oct 30 22:03:48.087733 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:48.087692    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:48.092882 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:48.092853    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:48.766462 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:48.766429    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:48.767544 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:48.767525    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:03:52.354593 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:03:52.354547    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:03:54.525081 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.525042    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"]
Oct 30 22:03:54.528613 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.528594    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:54.584723 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.584689    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"]
Oct 30 22:03:54.632333 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.632300    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-197a40\""
Oct 30 22:03:54.632710 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.632685    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-d27a8917-dockercfg-mxtkz\""
Oct 30 22:03:54.638368 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.638308    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzdsx\" (UniqueName: \"kubernetes.io/projected/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-kube-api-access-qzdsx\") pod \"payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n\" (UID: \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:54.638767 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.638747    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n\" (UID: \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:54.677072 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.677040    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"]
Oct 30 22:03:54.680815 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.680798    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:54.739547 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.739513    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"]
Oct 30 22:03:54.739808 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.739782    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c47887-8e5c-4a3b-b219-bf3c6ef57557-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k\" (UID: \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:54.739960 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.739881    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n\" (UID: \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:54.739960 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.739938    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgcqk\" (UniqueName: \"kubernetes.io/projected/78c47887-8e5c-4a3b-b219-bf3c6ef57557-kube-api-access-mgcqk\") pod \"payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k\" (UID: \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:54.740101 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.739966    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzdsx\" (UniqueName: \"kubernetes.io/projected/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-kube-api-access-qzdsx\") pod \"payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n\" (UID: \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:54.740832 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.740806    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n\" (UID: \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:54.799693 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.799613    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzdsx\" (UniqueName: \"kubernetes.io/projected/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-kube-api-access-qzdsx\") pod \"payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n\" (UID: \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:54.840579 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.840533    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgcqk\" (UniqueName: \"kubernetes.io/projected/78c47887-8e5c-4a3b-b219-bf3c6ef57557-kube-api-access-mgcqk\") pod \"payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k\" (UID: \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:54.840806 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.840645    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c47887-8e5c-4a3b-b219-bf3c6ef57557-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k\" (UID: \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:54.841408 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.841388    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c47887-8e5c-4a3b-b219-bf3c6ef57557-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k\" (UID: \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:54.842296 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.842277    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:54.900467 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.900406    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgcqk\" (UniqueName: \"kubernetes.io/projected/78c47887-8e5c-4a3b-b219-bf3c6ef57557-kube-api-access-mgcqk\") pod \"payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k\" (UID: \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:54.993925 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:54.993889    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:55.220820 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.220443    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"]
Oct 30 22:03:55.369166 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.369140    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"]
Oct 30 22:03:55.370694 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:03:55.370644    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c47887_8e5c_4a3b_b219_bf3c6ef57557.slice/crio-86b5aefefd141a9d63a1d23db2b09ae546b8a3bd995b2f997e2f6d27f62af54a WatchSource:0}: Error finding container 86b5aefefd141a9d63a1d23db2b09ae546b8a3bd995b2f997e2f6d27f62af54a: Status 404 returned error can't find the container with id 86b5aefefd141a9d63a1d23db2b09ae546b8a3bd995b2f997e2f6d27f62af54a
Oct 30 22:03:55.808161 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.808080    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k" event={"ID":"78c47887-8e5c-4a3b-b219-bf3c6ef57557","Type":"ContainerStarted","Data":"2c73ec3b7fb755f23dcbb0c02b6563220e080bd02d7e955562e9f6c6fb283bce"}
Oct 30 22:03:55.808680 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.808334    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:03:55.808680 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.808365    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k" event={"ID":"78c47887-8e5c-4a3b-b219-bf3c6ef57557","Type":"ContainerStarted","Data":"86b5aefefd141a9d63a1d23db2b09ae546b8a3bd995b2f997e2f6d27f62af54a"}
Oct 30 22:03:55.809848 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.809816    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n" event={"ID":"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c","Type":"ContainerStarted","Data":"1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2"}
Oct 30 22:03:55.809955 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.809868    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n" event={"ID":"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c","Type":"ContainerStarted","Data":"621bf1264899f373866c553c54fa6ae60552af774297be3144e575006b497445"}
Oct 30 22:03:55.809955 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.809896    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:03:55.924053 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:03:55.923994    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k" podStartSLOduration=1.923980071 podStartE2EDuration="1.923980071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:55.922039365 +0000 UTC m=+3091.793158485" watchObservedRunningTime="2026-10-30 22:03:55.923980071 +0000 UTC m=+3091.795099190"
Oct 30 22:04:06.818388 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:06.818355    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:04:06.818861 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:06.818534    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:04:06.939420 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:06.939363    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n" podStartSLOduration=12.939349052 podStartE2EDuration="12.939349052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:56.044742021 +0000 UTC m=+3091.915861158" watchObservedRunningTime="2026-10-30 22:04:06.939349052 +0000 UTC m=+3102.810468171"
Oct 30 22:04:13.926076 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:13.926044    5786 generic.go:356] "Generic (PLEG): container finished" podID="673ad515-c52c-48a5-8d24-b089ce633478" containerID="79356bf06ca1f1d2ab64338681839b5fe35081eaf1f549e266c0193655ea5224" exitCode=0
Oct 30 22:04:13.926542 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:13.926099    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" event={"ID":"673ad515-c52c-48a5-8d24-b089ce633478","Type":"ContainerDied","Data":"79356bf06ca1f1d2ab64338681839b5fe35081eaf1f549e266c0193655ea5224"}
Oct 30 22:04:13.926542 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:13.926517    5786 scope.go:122] "RemoveContainer" containerID="79356bf06ca1f1d2ab64338681839b5fe35081eaf1f549e266c0193655ea5224"
Oct 30 22:04:13.927821 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:13.927789    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:04:13.927880 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:13.927867    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:04:13.927920 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:13.927906    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:04:14.933521 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:14.933469    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" event={"ID":"673ad515-c52c-48a5-8d24-b089ce633478","Type":"ContainerStarted","Data":"1d65f8cdc3ea7c66afe0615eb0d23543060c722e97725e5411f08936dbc4062e"}
Oct 30 22:04:15.087462 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:15.087419    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:04:15.088755 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:15.088727    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused" start-of-body=
Oct 30 22:04:15.088885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:15.088777    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused"
Oct 30 22:04:16.087890 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:16.087859    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused" start-of-body=
Oct 30 22:04:16.088345 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:16.087906    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused"
Oct 30 22:04:17.087082 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:17.087039    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused" start-of-body=
Oct 30 22:04:17.087350 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:17.087102    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused"
Oct 30 22:04:18.062266 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:18.058586    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"]
Oct 30 22:04:18.087033 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:18.086996    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused" start-of-body=
Oct 30 22:04:18.087254 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:18.087054    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused"
Oct 30 22:04:18.102993 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:18.102965    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-4zbtb"]
Oct 30 22:04:18.378136 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:18.378030    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7462440c-9ee9-47dd-b623-ff3351aad152" path="/var/lib/kubelet/pods/7462440c-9ee9-47dd-b623-ff3351aad152/volumes"
Oct 30 22:04:18.772331 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:18.772294    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:04:19.088042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:19.087957    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused" start-of-body=
Oct 30 22:04:19.088042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:19.088010    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.236:15021/healthz/ready\": dial tcp 10.134.0.236:15021: connect: connection refused"
Oct 30 22:04:20.091550 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:20.091518    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:04:20.092581 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:20.092565    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:04:22.352923 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:04:22.352878    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:04:27.851676 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:27.851642    5786 scope.go:122] "RemoveContainer" containerID="a9d4366ac07b2cead4cd23f367941fcd99501ccd2e3f294818a7629b52ae084d"
Oct 30 22:04:27.862412 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:27.862363    5786 scope.go:122] "RemoveContainer" containerID="38ea0238707eb652740bbb41df01dcf818a2da392310a93144f1db7c2029750e"
Oct 30 22:04:27.871338 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:27.871321    5786 scope.go:122] "RemoveContainer" containerID="39805d76434364b4d59fb7c306edb66e63c56f095bf03e17ee5e107dad9fad31"
Oct 30 22:04:52.353681 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:04:52.353643    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:04:56.052890 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:56.052856    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"]
Oct 30 22:04:56.056269 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:56.056240    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-p4r9x"]
Oct 30 22:04:56.357998 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:04:56.357923    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3e9dcc0-92b0-49ee-8f4f-a862632036c4" path="/var/lib/kubelet/pods/e3e9dcc0-92b0-49ee-8f4f-a862632036c4/volumes"
Oct 30 22:05:03.636807 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.636722    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"]
Oct 30 22:05:03.640176 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.640159    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.691755 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.691716    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:05:03.741131 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.741099    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:05:03.741369 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.741172    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:05:03.741369 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.741356    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:05:03.743614 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.743588    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"]
Oct 30 22:05:03.817137 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.817100    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.817379 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.817278    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmsz6\" (UniqueName: \"kubernetes.io/projected/209f5887-36b8-48c6-8427-09a2376ddca7-kube-api-access-zmsz6\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.817379 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.817312    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.918075 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.918046    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.918289 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.918164    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.918289 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.918262    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmsz6\" (UniqueName: \"kubernetes.io/projected/209f5887-36b8-48c6-8427-09a2376ddca7-kube-api-access-zmsz6\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.918715 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.918691    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.920619 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.920597    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:03.974466 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:03.974442    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmsz6\" (UniqueName: \"kubernetes.io/projected/209f5887-36b8-48c6-8427-09a2376ddca7-kube-api-access-zmsz6\") pod \"maas-api-revoke-sub-f2ebebd5-vfvr7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:04.253029 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:04.252951    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:04.632290 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:04.632192    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"]
Oct 30 22:05:04.635381 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:05:04.635341    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209f5887_36b8_48c6_8427_09a2376ddca7.slice/crio-ddb4d2ec91b0aa13e8d23a6febccc67c1a007d840bcf2d93b2466449eeb39c1f WatchSource:0}: Error finding container ddb4d2ec91b0aa13e8d23a6febccc67c1a007d840bcf2d93b2466449eeb39c1f: Status 404 returned error can't find the container with id ddb4d2ec91b0aa13e8d23a6febccc67c1a007d840bcf2d93b2466449eeb39c1f
Oct 30 22:05:05.221811 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:05.221770    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7" event={"ID":"209f5887-36b8-48c6-8427-09a2376ddca7","Type":"ContainerStarted","Data":"92a90defae65244638ba5d831ab59cf4b5dd710a358224c45fdc7375e6633676"}
Oct 30 22:05:05.222251 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:05.221833    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7" event={"ID":"209f5887-36b8-48c6-8427-09a2376ddca7","Type":"ContainerStarted","Data":"ddb4d2ec91b0aa13e8d23a6febccc67c1a007d840bcf2d93b2466449eeb39c1f"}
Oct 30 22:05:05.335999 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:05.335949    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7" podStartSLOduration=2.335933773 podStartE2EDuration="2.335933773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:05.334628751 +0000 UTC m=+3161.205747870" watchObservedRunningTime="2026-10-30 22:05:05.335933773 +0000 UTC m=+3161.207052891"
Oct 30 22:05:06.228717 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:06.228683    5786 generic.go:356] "Generic (PLEG): container finished" podID="209f5887-36b8-48c6-8427-09a2376ddca7" containerID="92a90defae65244638ba5d831ab59cf4b5dd710a358224c45fdc7375e6633676" exitCode=0
Oct 30 22:05:06.229140 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:06.228732    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7" event={"ID":"209f5887-36b8-48c6-8427-09a2376ddca7","Type":"ContainerDied","Data":"92a90defae65244638ba5d831ab59cf4b5dd710a358224c45fdc7375e6633676"}
Oct 30 22:05:07.367266 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.367244    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:07.453882 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.453851    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-xks-ca\") pod \"209f5887-36b8-48c6-8427-09a2376ddca7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") "
Oct 30 22:05:07.454086 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.453908    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/209f5887-36b8-48c6-8427-09a2376ddca7-kube-api-access-zmsz6\" (UniqueName: \"kubernetes.io/projected/209f5887-36b8-48c6-8427-09a2376ddca7-kube-api-access-zmsz6\") pod \"209f5887-36b8-48c6-8427-09a2376ddca7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") "
Oct 30 22:05:07.454086 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.454006    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-openshift-ca\") pod \"209f5887-36b8-48c6-8427-09a2376ddca7\" (UID: \"209f5887-36b8-48c6-8427-09a2376ddca7\") "
Oct 30 22:05:07.454379 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.454351    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-openshift-ca" pod "209f5887-36b8-48c6-8427-09a2376ddca7" (UID: "209f5887-36b8-48c6-8427-09a2376ddca7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:07.456151 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.456128    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-xks-ca" pod "209f5887-36b8-48c6-8427-09a2376ddca7" (UID: "209f5887-36b8-48c6-8427-09a2376ddca7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:05:07.456247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.456181    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/209f5887-36b8-48c6-8427-09a2376ddca7-kube-api-access-zmsz6" pod "209f5887-36b8-48c6-8427-09a2376ddca7" (UID: "209f5887-36b8-48c6-8427-09a2376ddca7"). InnerVolumeSpecName "kube-api-access-zmsz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:07.555669 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.555594    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:07.555669 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.555620    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/209f5887-36b8-48c6-8427-09a2376ddca7-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:07.555669 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:07.555630    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmsz6\" (UniqueName: \"kubernetes.io/projected/209f5887-36b8-48c6-8427-09a2376ddca7-kube-api-access-zmsz6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:08.240222 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:08.240170    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"
Oct 30 22:05:08.240222 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:08.240199    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7" event={"ID":"209f5887-36b8-48c6-8427-09a2376ddca7","Type":"ContainerDied","Data":"ddb4d2ec91b0aa13e8d23a6febccc67c1a007d840bcf2d93b2466449eeb39c1f"}
Oct 30 22:05:08.240439 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:08.240268    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddb4d2ec91b0aa13e8d23a6febccc67c1a007d840bcf2d93b2466449eeb39c1f"
Oct 30 22:05:15.859375 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.859344    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"]
Oct 30 22:05:15.859812 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.859799    5786 cpu_manager.go:414] "containerMap: removing container" podUID="209f5887-36b8-48c6-8427-09a2376ddca7" containerName="revoke-keys"
Oct 30 22:05:15.859856 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.859812    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="209f5887-36b8-48c6-8427-09a2376ddca7" containerName="revoke-keys"
Oct 30 22:05:15.859889 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.859878    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="209f5887-36b8-48c6-8427-09a2376ddca7" containerName="revoke-keys"
Oct 30 22:05:15.863189 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.863174    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:15.941399 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.941359    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94kv8\" (UniqueName: \"kubernetes.io/projected/d582c6ba-06b8-41ee-b9c8-8adca74856c0-kube-api-access-94kv8\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:15.941601 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.941427    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:15.941677 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.941620    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:15.965941 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.965917    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:05:15.966092 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.965954    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:05:15.966092 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.966045    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:05:15.966092 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.966076    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:05:15.968009 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:15.967988    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"]
Oct 30 22:05:16.043015 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:16.042987    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:16.043125 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:16.043076    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94kv8\" (UniqueName: \"kubernetes.io/projected/d582c6ba-06b8-41ee-b9c8-8adca74856c0-kube-api-access-94kv8\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:16.043125 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:16.043115    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:16.043658 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:16.043638    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:16.045617 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:16.045592    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:16.099943 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:16.099916    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94kv8\" (UniqueName: \"kubernetes.io/projected/d582c6ba-06b8-41ee-b9c8-8adca74856c0-kube-api-access-94kv8\") pod \"maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:16.176126 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:16.176106    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:16.548552 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:16.548530    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"]
Oct 30 22:05:16.550883 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:05:16.550856    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd582c6ba_06b8_41ee_b9c8_8adca74856c0.slice/crio-f10f9b4030d9ca18320b31e42660be993d2b7bd4e3d796c6a37d38843ad598a0 WatchSource:0}: Error finding container f10f9b4030d9ca18320b31e42660be993d2b7bd4e3d796c6a37d38843ad598a0: Status 404 returned error can't find the container with id f10f9b4030d9ca18320b31e42660be993d2b7bd4e3d796c6a37d38843ad598a0
Oct 30 22:05:17.290823 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:17.290786    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb" event={"ID":"d582c6ba-06b8-41ee-b9c8-8adca74856c0","Type":"ContainerStarted","Data":"e40c3e9fb0e64d9727fd86f5c121084cf22538cc3653e08b47d1cbc478d06ca7"}
Oct 30 22:05:17.291317 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:17.290844    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb" event={"ID":"d582c6ba-06b8-41ee-b9c8-8adca74856c0","Type":"ContainerStarted","Data":"f10f9b4030d9ca18320b31e42660be993d2b7bd4e3d796c6a37d38843ad598a0"}
Oct 30 22:05:17.406436 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:17.406392    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb" podStartSLOduration=2.406377366 podStartE2EDuration="2.406377366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:17.403749217 +0000 UTC m=+3173.274868334" watchObservedRunningTime="2026-10-30 22:05:17.406377366 +0000 UTC m=+3173.277496479"
Oct 30 22:05:18.297219 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:18.297180    5786 generic.go:356] "Generic (PLEG): container finished" podID="d582c6ba-06b8-41ee-b9c8-8adca74856c0" containerID="e40c3e9fb0e64d9727fd86f5c121084cf22538cc3653e08b47d1cbc478d06ca7" exitCode=0
Oct 30 22:05:18.297626 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:18.297238    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb" event={"ID":"d582c6ba-06b8-41ee-b9c8-8adca74856c0","Type":"ContainerDied","Data":"e40c3e9fb0e64d9727fd86f5c121084cf22538cc3653e08b47d1cbc478d06ca7"}
Oct 30 22:05:19.434661 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.434638    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:19.479307 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.479281    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d582c6ba-06b8-41ee-b9c8-8adca74856c0-kube-api-access-94kv8\" (UniqueName: \"kubernetes.io/projected/d582c6ba-06b8-41ee-b9c8-8adca74856c0-kube-api-access-94kv8\") pod \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") "
Oct 30 22:05:19.479462 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.479357    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-xks-ca\") pod \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") "
Oct 30 22:05:19.479462 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.479438    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-openshift-ca\") pod \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\" (UID: \"d582c6ba-06b8-41ee-b9c8-8adca74856c0\") "
Oct 30 22:05:19.479774 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.479749    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-openshift-ca" pod "d582c6ba-06b8-41ee-b9c8-8adca74856c0" (UID: "d582c6ba-06b8-41ee-b9c8-8adca74856c0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:19.481531 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.481503    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-xks-ca" pod "d582c6ba-06b8-41ee-b9c8-8adca74856c0" (UID: "d582c6ba-06b8-41ee-b9c8-8adca74856c0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:05:19.481626 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.481525    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d582c6ba-06b8-41ee-b9c8-8adca74856c0-kube-api-access-94kv8" pod "d582c6ba-06b8-41ee-b9c8-8adca74856c0" (UID: "d582c6ba-06b8-41ee-b9c8-8adca74856c0"). InnerVolumeSpecName "kube-api-access-94kv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:19.581365 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.581292    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94kv8\" (UniqueName: \"kubernetes.io/projected/d582c6ba-06b8-41ee-b9c8-8adca74856c0-kube-api-access-94kv8\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:19.581365 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.581320    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:19.581365 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:19.581329    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d582c6ba-06b8-41ee-b9c8-8adca74856c0-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:20.308785 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:20.308752    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"
Oct 30 22:05:20.308956 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:20.308784    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb" event={"ID":"d582c6ba-06b8-41ee-b9c8-8adca74856c0","Type":"ContainerDied","Data":"f10f9b4030d9ca18320b31e42660be993d2b7bd4e3d796c6a37d38843ad598a0"}
Oct 30 22:05:20.308956 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:20.308836    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f10f9b4030d9ca18320b31e42660be993d2b7bd4e3d796c6a37d38843ad598a0"
Oct 30 22:05:22.353868 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:05:22.353835    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:05:27.048418 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:27.048376    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"]
Oct 30 22:05:27.097157 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:27.097122    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-hqmfk"]
Oct 30 22:05:27.350448 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:27.350375    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"]
Oct 30 22:05:27.439844 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:27.439815    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"]
Oct 30 22:05:27.902405 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:27.902373    5786 scope.go:122] "RemoveContainer" containerID="ca66245c268a46cf9a6fcde9d054546b686bf9b057e2b2767c4d48f761f604a0"
Oct 30 22:05:27.911653 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:27.911633    5786 scope.go:122] "RemoveContainer" containerID="e2eea57fe98d9794a101e893dd469d1dc1130b941724a5776f3cd191e1d700f0"
Oct 30 22:05:28.358554 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:28.358514    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5c71e6e-8806-4884-8329-987e224d23b2" path="/var/lib/kubelet/pods/e5c71e6e-8806-4884-8329-987e224d23b2/volumes"
Oct 30 22:05:32.351480 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:32.351440    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k" podUID="78c47887-8e5c-4a3b-b219-bf3c6ef57557" containerName="payload-processing" containerID="cri-o://2c73ec3b7fb755f23dcbb0c02b6563220e080bd02d7e955562e9f6c6fb283bce" gracePeriod=25
Oct 30 22:05:32.440200 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:32.440162    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n" podUID="2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" containerName="payload-pre-processing" containerID="cri-o://1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2" gracePeriod=25
Oct 30 22:05:36.437942 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:36.437904    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"]
Oct 30 22:05:36.486047 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:36.486015    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-197a40-wl2sb"]
Oct 30 22:05:36.817703 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:36.817619    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n" podUID="2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:05:36.817703 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:36.817664    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k" podUID="78c47887-8e5c-4a3b-b219-bf3c6ef57557" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:05:38.358059 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:38.358030    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d582c6ba-06b8-41ee-b9c8-8adca74856c0" path="/var/lib/kubelet/pods/d582c6ba-06b8-41ee-b9c8-8adca74856c0/volumes"
Oct 30 22:05:46.817124 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:46.817087    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k" podUID="78c47887-8e5c-4a3b-b219-bf3c6ef57557" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:05:46.817543 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:46.817094    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n" podUID="2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:05:52.354247 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:05:52.354195    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:05:52.496505 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.496476    5786 generic.go:356] "Generic (PLEG): container finished" podID="78c47887-8e5c-4a3b-b219-bf3c6ef57557" containerID="2c73ec3b7fb755f23dcbb0c02b6563220e080bd02d7e955562e9f6c6fb283bce" exitCode=0
Oct 30 22:05:52.496637 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.496520    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k" event={"ID":"78c47887-8e5c-4a3b-b219-bf3c6ef57557","Type":"ContainerDied","Data":"2c73ec3b7fb755f23dcbb0c02b6563220e080bd02d7e955562e9f6c6fb283bce"}
Oct 30 22:05:52.695598 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.695573    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:05:52.717650 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.717625    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:05:52.802328 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.802300    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-plugins-config-volume\") pod \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\" (UID: \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\") "
Oct 30 22:05:52.802491 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.802349    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78c47887-8e5c-4a3b-b219-bf3c6ef57557-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c47887-8e5c-4a3b-b219-bf3c6ef57557-plugins-config-volume\") pod \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\" (UID: \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\") "
Oct 30 22:05:52.802491 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.802379    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-kube-api-access-qzdsx\" (UniqueName: \"kubernetes.io/projected/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-kube-api-access-qzdsx\") pod \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\" (UID: \"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c\") "
Oct 30 22:05:52.802491 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.802425    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78c47887-8e5c-4a3b-b219-bf3c6ef57557-kube-api-access-mgcqk\" (UniqueName: \"kubernetes.io/projected/78c47887-8e5c-4a3b-b219-bf3c6ef57557-kube-api-access-mgcqk\") pod \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\" (UID: \"78c47887-8e5c-4a3b-b219-bf3c6ef57557\") "
Oct 30 22:05:52.802743 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.802714    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-plugins-config-volume" pod "2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" (UID: "2fc21cc5-c8e9-4cdc-9f28-12eeda98060c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:52.802873 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.802717    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c47887-8e5c-4a3b-b219-bf3c6ef57557-plugins-config-volume" pod "78c47887-8e5c-4a3b-b219-bf3c6ef57557" (UID: "78c47887-8e5c-4a3b-b219-bf3c6ef57557"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:52.804648 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.804630    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-kube-api-access-qzdsx" pod "2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" (UID: "2fc21cc5-c8e9-4cdc-9f28-12eeda98060c"). InnerVolumeSpecName "kube-api-access-qzdsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:52.804726 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.804668    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78c47887-8e5c-4a3b-b219-bf3c6ef57557-kube-api-access-mgcqk" pod "78c47887-8e5c-4a3b-b219-bf3c6ef57557" (UID: "78c47887-8e5c-4a3b-b219-bf3c6ef57557"). InnerVolumeSpecName "kube-api-access-mgcqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:52.903682 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.903608    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgcqk\" (UniqueName: \"kubernetes.io/projected/78c47887-8e5c-4a3b-b219-bf3c6ef57557-kube-api-access-mgcqk\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.903682 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.903634    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.903682 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.903645    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c47887-8e5c-4a3b-b219-bf3c6ef57557-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.903682 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:52.903653    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzdsx\" (UniqueName: \"kubernetes.io/projected/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c-kube-api-access-qzdsx\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:53.368144 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.368108    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"]
Oct 30 22:05:53.370121 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.369843    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy" containerID="cri-o://1d65f8cdc3ea7c66afe0615eb0d23543060c722e97725e5411f08936dbc4062e" gracePeriod=30
Oct 30 22:05:53.503166 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.503130    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k" event={"ID":"78c47887-8e5c-4a3b-b219-bf3c6ef57557","Type":"ContainerDied","Data":"86b5aefefd141a9d63a1d23db2b09ae546b8a3bd995b2f997e2f6d27f62af54a"}
Oct 30 22:05:53.503166 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.503159    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"
Oct 30 22:05:53.503427 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.503234    5786 scope.go:122] "RemoveContainer" containerID="2c73ec3b7fb755f23dcbb0c02b6563220e080bd02d7e955562e9f6c6fb283bce"
Oct 30 22:05:53.504540 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.504521    5786 generic.go:356] "Generic (PLEG): container finished" podID="2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" containerID="1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2" exitCode=0
Oct 30 22:05:53.504631 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.504565    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n" event={"ID":"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c","Type":"ContainerDied","Data":"1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2"}
Oct 30 22:05:53.504631 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.504586    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"
Oct 30 22:05:53.504631 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.504618    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n" event={"ID":"2fc21cc5-c8e9-4cdc-9f28-12eeda98060c","Type":"ContainerDied","Data":"621bf1264899f373866c553c54fa6ae60552af774297be3144e575006b497445"}
Oct 30 22:05:53.515952 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.515917    5786 scope.go:122] "RemoveContainer" containerID="1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2"
Oct 30 22:05:53.524693 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.524673    5786 scope.go:122] "RemoveContainer" containerID="1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2"
Oct 30 22:05:53.524954 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:05:53.524936    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2\": container with ID starting with 1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2 not found: ID does not exist" containerID="1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2"
Oct 30 22:05:53.525015 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.524966    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2"} err="failed to get container status \"1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2\": rpc error: code = NotFound desc = could not find container \"1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2\": container with ID starting with 1c61a0256e2d062eec15afc203625ee01673d373a8ebbb4db3aede24ef19d7b2 not found: ID does not exist"
Oct 30 22:05:53.666829 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.666790    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"]
Oct 30 22:05:53.670552 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.670525    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-197a40-d4c77qdf7n"]
Oct 30 22:05:53.777674 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.777644    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"]
Oct 30 22:05:53.781166 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:53.781131    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-197a40-59cc69f98lcm7k"]
Oct 30 22:05:54.359265 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:54.359231    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" path="/var/lib/kubelet/pods/2fc21cc5-c8e9-4cdc-9f28-12eeda98060c/volumes"
Oct 30 22:05:54.359629 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:54.359616    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78c47887-8e5c-4a3b-b219-bf3c6ef57557" path="/var/lib/kubelet/pods/78c47887-8e5c-4a3b-b219-bf3c6ef57557/volumes"
Oct 30 22:05:58.535506 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.535477    5786 generic.go:356] "Generic (PLEG): container finished" podID="673ad515-c52c-48a5-8d24-b089ce633478" containerID="1d65f8cdc3ea7c66afe0615eb0d23543060c722e97725e5411f08936dbc4062e" exitCode=0
Oct 30 22:05:58.535902 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.535557    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" event={"ID":"673ad515-c52c-48a5-8d24-b089ce633478","Type":"ContainerDied","Data":"1d65f8cdc3ea7c66afe0615eb0d23543060c722e97725e5411f08936dbc4062e"}
Oct 30 22:05:58.535902 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.535633    5786 scope.go:122] "RemoveContainer" containerID="79356bf06ca1f1d2ab64338681839b5fe35081eaf1f549e266c0193655ea5224"
Oct 30 22:05:58.626098 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.626076    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:05:58.668928 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.668901    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-credential-socket\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.668928 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.668935    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-certs\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.668928 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.668954    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/673ad515-c52c-48a5-8d24-b089ce633478-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673ad515-c52c-48a5-8d24-b089ce633478-istiod-ca-cert\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.669277 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669002    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-socket\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.669277 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669113    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-envoy\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.669277 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669198    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-istio-token\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-istio-token\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.669277 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669247    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-credential-socket" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:58.669277 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669269    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-certs" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:58.669522 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669275    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/673ad515-c52c-48a5-8d24-b089ce633478-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673ad515-c52c-48a5-8d24-b089ce633478-istio-podinfo\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.669522 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669390    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-socket" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:58.669522 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669414    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-kube-api-access-8zn5j\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-kube-api-access-8zn5j\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.669522 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669460    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-data\") pod \"673ad515-c52c-48a5-8d24-b089ce633478\" (UID: \"673ad515-c52c-48a5-8d24-b089ce633478\") "
Oct 30 22:05:58.669522 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669461    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/673ad515-c52c-48a5-8d24-b089ce633478-istiod-ca-cert" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:58.669798 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669780    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-data" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:58.670074 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.669990    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:58.670149 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.670080    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:58.670149 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.670096    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:58.670149 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.670109    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673ad515-c52c-48a5-8d24-b089ce633478-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:58.670149 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.670122    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:58.671482 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.671459    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-envoy" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:58.671589 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.671568    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/673ad515-c52c-48a5-8d24-b089ce633478-istio-podinfo" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:58.671630 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.671584    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-istio-token" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:58.671630 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.671595    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-kube-api-access-8zn5j" pod "673ad515-c52c-48a5-8d24-b089ce633478" (UID: "673ad515-c52c-48a5-8d24-b089ce633478"). InnerVolumeSpecName "kube-api-access-8zn5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:58.770773 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.770747    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673ad515-c52c-48a5-8d24-b089ce633478-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:58.770773 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.770772    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:58.770960 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.770781    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673ad515-c52c-48a5-8d24-b089ce633478-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:58.770960 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:58.770789    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zn5j\" (UniqueName: \"kubernetes.io/projected/673ad515-c52c-48a5-8d24-b089ce633478-kube-api-access-8zn5j\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:59.542147 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:59.542116    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"
Oct 30 22:05:59.542653 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:59.542156    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz" event={"ID":"673ad515-c52c-48a5-8d24-b089ce633478","Type":"ContainerDied","Data":"764a030038194d0b13bdf82043219af3c5385aa2654a78cc52ff5ed67d95204a"}
Oct 30 22:05:59.542653 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:59.542249    5786 scope.go:122] "RemoveContainer" containerID="1d65f8cdc3ea7c66afe0615eb0d23543060c722e97725e5411f08936dbc4062e"
Oct 30 22:05:59.753389 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:59.753357    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"]
Oct 30 22:05:59.773409 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:05:59.773379    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-197a40-openshift-default-697f68c79cdrgdz"]
Oct 30 22:06:00.358115 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:00.358084    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="673ad515-c52c-48a5-8d24-b089ce633478" path="/var/lib/kubelet/pods/673ad515-c52c-48a5-8d24-b089ce633478/volumes"
Oct 30 22:06:04.957775 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:04.957740    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"]
Oct 30 22:06:05.006870 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.006838    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-84h7b"]
Oct 30 22:06:05.080526 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080493    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"]
Oct 30 22:06:05.080912 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080900    5786 cpu_manager.go:414] "containerMap: removing container" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy"
Oct 30 22:06:05.080961 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080913    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy"
Oct 30 22:06:05.080961 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080931    5786 cpu_manager.go:414] "containerMap: removing container" podUID="d582c6ba-06b8-41ee-b9c8-8adca74856c0" containerName="revoke-keys"
Oct 30 22:06:05.080961 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080936    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d582c6ba-06b8-41ee-b9c8-8adca74856c0" containerName="revoke-keys"
Oct 30 22:06:05.080961 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080956    5786 cpu_manager.go:414] "containerMap: removing container" podUID="2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" containerName="payload-pre-processing"
Oct 30 22:06:05.080961 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080961    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" containerName="payload-pre-processing"
Oct 30 22:06:05.081111 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080980    5786 cpu_manager.go:414] "containerMap: removing container" podUID="78c47887-8e5c-4a3b-b219-bf3c6ef57557" containerName="payload-processing"
Oct 30 22:06:05.081111 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.080985    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78c47887-8e5c-4a3b-b219-bf3c6ef57557" containerName="payload-processing"
Oct 30 22:06:05.081111 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.081040    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fc21cc5-c8e9-4cdc-9f28-12eeda98060c" containerName="payload-pre-processing"
Oct 30 22:06:05.081111 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.081051    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="78c47887-8e5c-4a3b-b219-bf3c6ef57557" containerName="payload-processing"
Oct 30 22:06:05.081111 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.081057    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy"
Oct 30 22:06:05.081111 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.081065    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy"
Oct 30 22:06:05.081111 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.081070    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="d582c6ba-06b8-41ee-b9c8-8adca74856c0" containerName="revoke-keys"
Oct 30 22:06:05.085643 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.085623    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.139550 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.139502    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:06:05.139550 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.139504    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:06:05.139781 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.139566    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:06:05.139781 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.139690    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:06:05.141322 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.141293    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"]
Oct 30 22:06:05.237659 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.237575    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.237659 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.237619    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtprt\" (UniqueName: \"kubernetes.io/projected/6f2b948c-d2dc-4750-87f4-33f9c0570a44-kube-api-access-dtprt\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.237862 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.237699    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.338639 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.338594    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.338858 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.338720    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.338858 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.338748    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtprt\" (UniqueName: \"kubernetes.io/projected/6f2b948c-d2dc-4750-87f4-33f9c0570a44-kube-api-access-dtprt\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.339427 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.339402    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.341024 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.341002    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.395099 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.395068    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtprt\" (UniqueName: \"kubernetes.io/projected/6f2b948c-d2dc-4750-87f4-33f9c0570a44-kube-api-access-dtprt\") pod \"maas-api-revoke-sub-3d4156f1-schnz\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.398893 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.398871    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:05.771503 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:05.771474    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"]
Oct 30 22:06:05.774779 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:06:05.774744    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f2b948c_d2dc_4750_87f4_33f9c0570a44.slice/crio-a27b8a7f68607d5cfc9ab00a3fecc537b3d3f67a5d1e6ef1c9383d196cd3dc8b WatchSource:0}: Error finding container a27b8a7f68607d5cfc9ab00a3fecc537b3d3f67a5d1e6ef1c9383d196cd3dc8b: Status 404 returned error can't find the container with id a27b8a7f68607d5cfc9ab00a3fecc537b3d3f67a5d1e6ef1c9383d196cd3dc8b
Oct 30 22:06:06.359955 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:06.359874    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d474d067-bf6f-40ef-bfd6-2b2402ddda15" path="/var/lib/kubelet/pods/d474d067-bf6f-40ef-bfd6-2b2402ddda15/volumes"
Oct 30 22:06:06.582531 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:06.582502    5786 generic.go:356] "Generic (PLEG): container finished" podID="6f2b948c-d2dc-4750-87f4-33f9c0570a44" containerID="10094d68e39f3bc6c80b63f78a6f0bda26e367fd352ae518e436668c58da6904" exitCode=0
Oct 30 22:06:06.582657 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:06.582535    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz" event={"ID":"6f2b948c-d2dc-4750-87f4-33f9c0570a44","Type":"ContainerDied","Data":"10094d68e39f3bc6c80b63f78a6f0bda26e367fd352ae518e436668c58da6904"}
Oct 30 22:06:06.582657 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:06.582572    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz" event={"ID":"6f2b948c-d2dc-4750-87f4-33f9c0570a44","Type":"ContainerStarted","Data":"a27b8a7f68607d5cfc9ab00a3fecc537b3d3f67a5d1e6ef1c9383d196cd3dc8b"}
Oct 30 22:06:07.721730 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.721709    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:07.865762 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.865674    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-openshift-ca\") pod \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") "
Oct 30 22:06:07.865762 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.865711    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f2b948c-d2dc-4750-87f4-33f9c0570a44-kube-api-access-dtprt\" (UniqueName: \"kubernetes.io/projected/6f2b948c-d2dc-4750-87f4-33f9c0570a44-kube-api-access-dtprt\") pod \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") "
Oct 30 22:06:07.866042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.865768    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-xks-ca\") pod \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\" (UID: \"6f2b948c-d2dc-4750-87f4-33f9c0570a44\") "
Oct 30 22:06:07.866106 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.866059    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-openshift-ca" pod "6f2b948c-d2dc-4750-87f4-33f9c0570a44" (UID: "6f2b948c-d2dc-4750-87f4-33f9c0570a44"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:06:07.866165 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.866139    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:07.867896 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.867871    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-xks-ca" pod "6f2b948c-d2dc-4750-87f4-33f9c0570a44" (UID: "6f2b948c-d2dc-4750-87f4-33f9c0570a44"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:06:07.867896 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.867882    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f2b948c-d2dc-4750-87f4-33f9c0570a44-kube-api-access-dtprt" pod "6f2b948c-d2dc-4750-87f4-33f9c0570a44" (UID: "6f2b948c-d2dc-4750-87f4-33f9c0570a44"). InnerVolumeSpecName "kube-api-access-dtprt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:07.966952 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.966927    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtprt\" (UniqueName: \"kubernetes.io/projected/6f2b948c-d2dc-4750-87f4-33f9c0570a44-kube-api-access-dtprt\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:07.966952 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:07.966951    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f2b948c-d2dc-4750-87f4-33f9c0570a44-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:08.050580 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:08.050552    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"]
Oct 30 22:06:08.056576 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:08.053360    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-zn5nm"]
Oct 30 22:06:08.358767 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:08.358738    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11007f5d-6d51-4243-aa19-ca15195cdf6a" path="/var/lib/kubelet/pods/11007f5d-6d51-4243-aa19-ca15195cdf6a/volumes"
Oct 30 22:06:08.593470 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:08.593445    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"
Oct 30 22:06:08.593645 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:08.593489    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz" event={"ID":"6f2b948c-d2dc-4750-87f4-33f9c0570a44","Type":"ContainerDied","Data":"a27b8a7f68607d5cfc9ab00a3fecc537b3d3f67a5d1e6ef1c9383d196cd3dc8b"}
Oct 30 22:06:08.593645 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:08.593539    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a27b8a7f68607d5cfc9ab00a3fecc537b3d3f67a5d1e6ef1c9383d196cd3dc8b"
Oct 30 22:06:22.354330 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:06:22.354297    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:06:27.939299 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:27.939272    5786 scope.go:122] "RemoveContainer" containerID="535a68b42c07490a34408e5e7425979e276637a78a723fea761fc53c9fec0362"
Oct 30 22:06:27.947950 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:06:27.947926    5786 scope.go:122] "RemoveContainer" containerID="12da0f305b968877a7423b48c0b6b42c7b55d2f1d53388bae86cc77a0ad9129c"
Oct 30 22:06:52.355382 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:06:52.355293    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:07:22.355059 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:07:22.355025    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:07:27.983783 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:07:27.983754    5786 scope.go:122] "RemoveContainer" containerID="b28dc1cc74cafe6b846a1dc63d1d9c32642937ef44a5bec1173ef6ef1e7d4b8f"
Oct 30 22:07:27.992836 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:07:27.992749    5786 scope.go:122] "RemoveContainer" containerID="b0efb7738705eb31abd89fc61eba1d3bf7263a2c149184dcaf999ef528a47ce7"
Oct 30 22:07:50.048381 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:07:50.048349    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"]
Oct 30 22:07:50.097427 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:07:50.097399    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kqhhl"]
Oct 30 22:07:50.359072 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:07:50.358996    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8acec2ee-7410-4c77-9ed1-d3495da3ece0" path="/var/lib/kubelet/pods/8acec2ee-7410-4c77-9ed1-d3495da3ece0/volumes"
Oct 30 22:07:52.353958 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:07:52.353920    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:08:00.060381 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:00.060297    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"]
Oct 30 22:08:00.109643 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:00.109616    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-h4zm4"]
Oct 30 22:08:00.358016 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:00.357934    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="737e56e0-5dd6-4965-9ee0-0a6ed20054d5" path="/var/lib/kubelet/pods/737e56e0-5dd6-4965-9ee0-0a6ed20054d5/volumes"
Oct 30 22:08:22.354795 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:08:22.354760    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:08:28.019224 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.019176    5786 scope.go:122] "RemoveContainer" containerID="6a2daefc5e1d119644f7b1ed5dd6e3693b3db89ae5bc6a3db51140c4b4df5078"
Oct 30 22:08:28.339189 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.339107    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"]
Oct 30 22:08:28.339682 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.339661    5786 cpu_manager.go:414] "containerMap: removing container" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy"
Oct 30 22:08:28.339789 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.339684    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="673ad515-c52c-48a5-8d24-b089ce633478" containerName="istio-proxy"
Oct 30 22:08:28.339789 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.339745    5786 cpu_manager.go:414] "containerMap: removing container" podUID="6f2b948c-d2dc-4750-87f4-33f9c0570a44" containerName="revoke-keys"
Oct 30 22:08:28.339789 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.339767    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f2b948c-d2dc-4750-87f4-33f9c0570a44" containerName="revoke-keys"
Oct 30 22:08:28.339959 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.339880    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f2b948c-d2dc-4750-87f4-33f9c0570a44" containerName="revoke-keys"
Oct 30 22:08:28.344195 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.344174    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.400134 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.400105    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"]
Oct 30 22:08:28.426753 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.426728    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.426935 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.426783    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h84bf\" (UniqueName: \"kubernetes.io/projected/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-kube-api-access-h84bf\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.426935 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.426812    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.446433 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.446405    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:08:28.446433 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.446430    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:08:28.446599 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.446445    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:08:28.446599 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.446452    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:08:28.528375 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.528339    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.528590 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.528402    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h84bf\" (UniqueName: \"kubernetes.io/projected/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-kube-api-access-h84bf\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.528590 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.528437    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.529169 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.529143    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.530883 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.530863    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.584584 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.584552    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h84bf\" (UniqueName: \"kubernetes.io/projected/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-kube-api-access-h84bf\") pod \"maas-api-revoke-sub-d576e68b-75vm4\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:28.657043 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:28.656976    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:29.028169 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:29.028143    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"]
Oct 30 22:08:29.030952 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:08:29.030925    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80b0d54b_734d_4c2d_9b5d_5415054ce9c3.slice/crio-39b164798e3739fa72e6f7da06c7ae32ef986a7f53a45243602b78979f05b14f WatchSource:0}: Error finding container 39b164798e3739fa72e6f7da06c7ae32ef986a7f53a45243602b78979f05b14f: Status 404 returned error can't find the container with id 39b164798e3739fa72e6f7da06c7ae32ef986a7f53a45243602b78979f05b14f
Oct 30 22:08:29.332317 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:29.332276    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4" event={"ID":"80b0d54b-734d-4c2d-9b5d-5415054ce9c3","Type":"ContainerStarted","Data":"709100e57714cd5d729cc4cc4e15fc257cf35fc431599b14fec69ef9c2209e3e"}
Oct 30 22:08:29.332517 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:29.332338    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4" event={"ID":"80b0d54b-734d-4c2d-9b5d-5415054ce9c3","Type":"ContainerStarted","Data":"39b164798e3739fa72e6f7da06c7ae32ef986a7f53a45243602b78979f05b14f"}
Oct 30 22:08:29.446169 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:29.446122    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4" podStartSLOduration=1.446107329 podStartE2EDuration="1.446107329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:29.445836006 +0000 UTC m=+3365.316955127" watchObservedRunningTime="2026-10-30 22:08:29.446107329 +0000 UTC m=+3365.317226449"
Oct 30 22:08:30.339267 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:30.339237    5786 generic.go:356] "Generic (PLEG): container finished" podID="80b0d54b-734d-4c2d-9b5d-5415054ce9c3" containerID="709100e57714cd5d729cc4cc4e15fc257cf35fc431599b14fec69ef9c2209e3e" exitCode=0
Oct 30 22:08:30.339735 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:30.339303    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4" event={"ID":"80b0d54b-734d-4c2d-9b5d-5415054ce9c3","Type":"ContainerDied","Data":"709100e57714cd5d729cc4cc4e15fc257cf35fc431599b14fec69ef9c2209e3e"}
Oct 30 22:08:31.477762 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.477740    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:31.557554 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.557525    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-openshift-ca\") pod \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") "
Oct 30 22:08:31.557717 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.557621    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-xks-ca\") pod \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") "
Oct 30 22:08:31.557789 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.557714    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-kube-api-access-h84bf\" (UniqueName: \"kubernetes.io/projected/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-kube-api-access-h84bf\") pod \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\" (UID: \"80b0d54b-734d-4c2d-9b5d-5415054ce9c3\") "
Oct 30 22:08:31.557923 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.557902    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-openshift-ca" pod "80b0d54b-734d-4c2d-9b5d-5415054ce9c3" (UID: "80b0d54b-734d-4c2d-9b5d-5415054ce9c3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:31.558110 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.558093    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:31.559853 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.559825    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-xks-ca" pod "80b0d54b-734d-4c2d-9b5d-5415054ce9c3" (UID: "80b0d54b-734d-4c2d-9b5d-5415054ce9c3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:08:31.559947 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.559904    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-kube-api-access-h84bf" pod "80b0d54b-734d-4c2d-9b5d-5415054ce9c3" (UID: "80b0d54b-734d-4c2d-9b5d-5415054ce9c3"). InnerVolumeSpecName "kube-api-access-h84bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:31.659142 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.659063    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h84bf\" (UniqueName: \"kubernetes.io/projected/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-kube-api-access-h84bf\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:31.659142 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:31.659091    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b0d54b-734d-4c2d-9b5d-5415054ce9c3-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:32.350437 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:32.350405    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"
Oct 30 22:08:32.350437 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:32.350417    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4" event={"ID":"80b0d54b-734d-4c2d-9b5d-5415054ce9c3","Type":"ContainerDied","Data":"39b164798e3739fa72e6f7da06c7ae32ef986a7f53a45243602b78979f05b14f"}
Oct 30 22:08:32.350716 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:32.350460    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39b164798e3739fa72e6f7da06c7ae32ef986a7f53a45243602b78979f05b14f"
Oct 30 22:08:52.355032 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:08:52.354991    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:08:57.220141 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.220107    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"]
Oct 30 22:08:57.220630 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.220614    5786 cpu_manager.go:414] "containerMap: removing container" podUID="80b0d54b-734d-4c2d-9b5d-5415054ce9c3" containerName="revoke-keys"
Oct 30 22:08:57.220678 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.220632    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80b0d54b-734d-4c2d-9b5d-5415054ce9c3" containerName="revoke-keys"
Oct 30 22:08:57.220714 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.220706    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="80b0d54b-734d-4c2d-9b5d-5415054ce9c3" containerName="revoke-keys"
Oct 30 22:08:57.223815 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.223799    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.274194 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.274152    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx7wn\" (UniqueName: \"kubernetes.io/projected/a976f22c-8b5a-4592-b143-6ac83fd20559-kube-api-access-tx7wn\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.274194 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.274195    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.274546 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.274398    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.276932 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.276910    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:08:57.277054 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.277036    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:08:57.326335 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.326302    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:08:57.326532 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.326496    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:08:57.328241 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.328201    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"]
Oct 30 22:08:57.376015 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.375972    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.376287 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.376057    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx7wn\" (UniqueName: \"kubernetes.io/projected/a976f22c-8b5a-4592-b143-6ac83fd20559-kube-api-access-tx7wn\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.376287 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.376076    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.376669 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.376644    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.378617 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.378590    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.432554 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.432529    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx7wn\" (UniqueName: \"kubernetes.io/projected/a976f22c-8b5a-4592-b143-6ac83fd20559-kube-api-access-tx7wn\") pod \"maas-api-revoke-sub-d2bc93bc-4csqs\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.537458 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.537369    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:08:57.911881 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:57.911798    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"]
Oct 30 22:08:57.915083 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:08:57.915045    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda976f22c_8b5a_4592_b143_6ac83fd20559.slice/crio-f7054f11f8b34b624fdf634ab66cd08ab063f7a860d3719462f9ab2bd3f650bf WatchSource:0}: Error finding container f7054f11f8b34b624fdf634ab66cd08ab063f7a860d3719462f9ab2bd3f650bf: Status 404 returned error can't find the container with id f7054f11f8b34b624fdf634ab66cd08ab063f7a860d3719462f9ab2bd3f650bf
Oct 30 22:08:58.496954 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:58.496921    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs" event={"ID":"a976f22c-8b5a-4592-b143-6ac83fd20559","Type":"ContainerStarted","Data":"505d3e7a9b76525d47fae877a6db6d9fb0a63bb87ca66f9d4e900ad0c6d56e27"}
Oct 30 22:08:58.497409 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:58.496973    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs" event={"ID":"a976f22c-8b5a-4592-b143-6ac83fd20559","Type":"ContainerStarted","Data":"f7054f11f8b34b624fdf634ab66cd08ab063f7a860d3719462f9ab2bd3f650bf"}
Oct 30 22:08:58.610624 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:58.610460    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs" podStartSLOduration=1.610441617 podStartE2EDuration="1.610441617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:58.609787434 +0000 UTC m=+3394.480906551" watchObservedRunningTime="2026-10-30 22:08:58.610441617 +0000 UTC m=+3394.481560734"
Oct 30 22:08:59.504059 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:59.504021    5786 generic.go:356] "Generic (PLEG): container finished" podID="a976f22c-8b5a-4592-b143-6ac83fd20559" containerID="505d3e7a9b76525d47fae877a6db6d9fb0a63bb87ca66f9d4e900ad0c6d56e27" exitCode=0
Oct 30 22:08:59.504546 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:08:59.504098    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs" event={"ID":"a976f22c-8b5a-4592-b143-6ac83fd20559","Type":"ContainerDied","Data":"505d3e7a9b76525d47fae877a6db6d9fb0a63bb87ca66f9d4e900ad0c6d56e27"}
Oct 30 22:09:00.665712 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.665682    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:09:00.810688 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.810604    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a976f22c-8b5a-4592-b143-6ac83fd20559-kube-api-access-tx7wn\" (UniqueName: \"kubernetes.io/projected/a976f22c-8b5a-4592-b143-6ac83fd20559-kube-api-access-tx7wn\") pod \"a976f22c-8b5a-4592-b143-6ac83fd20559\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") "
Oct 30 22:09:00.810688 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.810667    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-openshift-ca\") pod \"a976f22c-8b5a-4592-b143-6ac83fd20559\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") "
Oct 30 22:09:00.810914 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.810786    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-xks-ca\") pod \"a976f22c-8b5a-4592-b143-6ac83fd20559\" (UID: \"a976f22c-8b5a-4592-b143-6ac83fd20559\") "
Oct 30 22:09:00.811149 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.811116    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-openshift-ca" pod "a976f22c-8b5a-4592-b143-6ac83fd20559" (UID: "a976f22c-8b5a-4592-b143-6ac83fd20559"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:00.813059 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.813034    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-xks-ca" pod "a976f22c-8b5a-4592-b143-6ac83fd20559" (UID: "a976f22c-8b5a-4592-b143-6ac83fd20559"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:00.813172 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.813152    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a976f22c-8b5a-4592-b143-6ac83fd20559-kube-api-access-tx7wn" pod "a976f22c-8b5a-4592-b143-6ac83fd20559" (UID: "a976f22c-8b5a-4592-b143-6ac83fd20559"). InnerVolumeSpecName "kube-api-access-tx7wn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:00.912615 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.912576    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx7wn\" (UniqueName: \"kubernetes.io/projected/a976f22c-8b5a-4592-b143-6ac83fd20559-kube-api-access-tx7wn\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:00.912615 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.912606    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:00.912615 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:00.912616    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a976f22c-8b5a-4592-b143-6ac83fd20559-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:01.515518 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:01.515487    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"
Oct 30 22:09:01.515711 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:01.515487    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs" event={"ID":"a976f22c-8b5a-4592-b143-6ac83fd20559","Type":"ContainerDied","Data":"f7054f11f8b34b624fdf634ab66cd08ab063f7a860d3719462f9ab2bd3f650bf"}
Oct 30 22:09:01.515711 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:01.515616    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7054f11f8b34b624fdf634ab66cd08ab063f7a860d3719462f9ab2bd3f650bf"
Oct 30 22:09:20.141009 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.140928    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"]
Oct 30 22:09:20.141472 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.141388    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a976f22c-8b5a-4592-b143-6ac83fd20559" containerName="revoke-keys"
Oct 30 22:09:20.141472 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.141400    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a976f22c-8b5a-4592-b143-6ac83fd20559" containerName="revoke-keys"
Oct 30 22:09:20.141561 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.141486    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a976f22c-8b5a-4592-b143-6ac83fd20559" containerName="revoke-keys"
Oct 30 22:09:20.144603 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.144583    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.196284 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.196254    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-dc7946-openshift-default-dockercfg-csps5\""
Oct 30 22:09:20.211969 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.211914    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-credential-socket\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.212157 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.212052    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-data\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.212157 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.212107    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-socket\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.212157 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.212131    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl72p\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-kube-api-access-tl72p\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.212358 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.212163    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-token\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.212421 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.212406    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istiod-ca-cert\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.212478 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.212440    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-envoy\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.212558 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.212541    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-podinfo\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.212617 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.212570    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-certs\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.249541 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.249489    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"]
Oct 30 22:09:20.313989 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.313950    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-data\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314198 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314003    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-socket\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314497 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314469    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-data\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314599 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314526    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-socket\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314599 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314534    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl72p\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-kube-api-access-tl72p\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314599 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314590    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-token\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314773 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314744    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istiod-ca-cert\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314832 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314774    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-envoy\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314832 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314817    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-podinfo\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314952 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314848    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-certs\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.314952 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.314881    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-credential-socket\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.315170 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.315148    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-credential-socket\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.315445 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.315398    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istiod-ca-cert\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.315800 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.315783    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-certs\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.329813 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.329643    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-envoy\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.333058 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.333025    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-podinfo\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.370881 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.370850    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl72p\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-kube-api-access-tl72p\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.372604 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.372575    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-token\") pod \"e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") " pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.457950 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.457835    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:20.887810 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.887785    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"]
Oct 30 22:09:20.890641 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:09:20.890607    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ba3e4c_c596_4a2c_8193_ccb21c5f3e2b.slice/crio-f897d230bd40374a43f7c3c8860b1b2786e3036d585b2906c0b02cb78adea19d WatchSource:0}: Error finding container f897d230bd40374a43f7c3c8860b1b2786e3036d585b2906c0b02cb78adea19d: Status 404 returned error can't find the container with id f897d230bd40374a43f7c3c8860b1b2786e3036d585b2906c0b02cb78adea19d
Oct 30 22:09:20.892935 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.892900    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:09:20.893025 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.892969    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:09:20.893025 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:20.893000    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:09:21.626346 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:21.626309    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" event={"ID":"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b","Type":"ContainerStarted","Data":"7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333"}
Oct 30 22:09:21.626867 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:21.626366    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" event={"ID":"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b","Type":"ContainerStarted","Data":"f897d230bd40374a43f7c3c8860b1b2786e3036d585b2906c0b02cb78adea19d"}
Oct 30 22:09:21.745946 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:21.745894    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" podStartSLOduration=1.745877738 podStartE2EDuration="1.745877738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:21.743007279 +0000 UTC m=+3417.614126411" watchObservedRunningTime="2026-10-30 22:09:21.745877738 +0000 UTC m=+3417.616996858"
Oct 30 22:09:22.354702 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:09:22.354654    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:09:22.458900 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:22.458869    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:22.463712 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:22.463686    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:22.631554 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:22.631471    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:22.632619 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:22.632598    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:28.040914 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.040877    5786 scope.go:122] "RemoveContainer" containerID="88c997265f0767e1344aba3a366bae335e259f56abdef54018b40233804fb822"
Oct 30 22:09:28.050127 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.050104    5786 scope.go:122] "RemoveContainer" containerID="0e6caf010fe17a6bd222cf0831b1289f4595bf40f246f0fd2f268f43bbde5980"
Oct 30 22:09:28.059465 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.059446    5786 scope.go:122] "RemoveContainer" containerID="17b3381830171744e4f5ec26005c9465439b8cbf5cc53528693130868713c390"
Oct 30 22:09:28.246866 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.246825    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"]
Oct 30 22:09:28.252084 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.252061    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:28.306676 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.306572    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw\" (UID: \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:28.306867 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.306755    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnsj6\" (UniqueName: \"kubernetes.io/projected/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-kube-api-access-jnsj6\") pod \"payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw\" (UID: \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:28.354419 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.354334    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-3a874e42-dockercfg-kffzq\""
Oct 30 22:09:28.354633 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.354444    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-dc7946\""
Oct 30 22:09:28.361347 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.361234    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"]
Oct 30 22:09:28.406582 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.406530    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"]
Oct 30 22:09:28.408187 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.407832    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw\" (UID: \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:28.408187 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.407966    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnsj6\" (UniqueName: \"kubernetes.io/projected/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-kube-api-access-jnsj6\") pod \"payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw\" (UID: \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:28.408887 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.408855    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw\" (UID: \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:28.410836 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.410807    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:28.465469 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.465438    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"]
Oct 30 22:09:28.467393 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.467360    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnsj6\" (UniqueName: \"kubernetes.io/projected/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-kube-api-access-jnsj6\") pod \"payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw\" (UID: \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:28.508898 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.508866    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gmjw\" (UniqueName: \"kubernetes.io/projected/11bc8568-9158-46b4-a967-63910dfb3026-kube-api-access-4gmjw\") pod \"payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m\" (UID: \"11bc8568-9158-46b4-a967-63910dfb3026\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:28.509082 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.508911    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11bc8568-9158-46b4-a967-63910dfb3026-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m\" (UID: \"11bc8568-9158-46b4-a967-63910dfb3026\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:28.565366 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.565273    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:28.610265 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.610227    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gmjw\" (UniqueName: \"kubernetes.io/projected/11bc8568-9158-46b4-a967-63910dfb3026-kube-api-access-4gmjw\") pod \"payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m\" (UID: \"11bc8568-9158-46b4-a967-63910dfb3026\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:28.610458 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.610284    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11bc8568-9158-46b4-a967-63910dfb3026-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m\" (UID: \"11bc8568-9158-46b4-a967-63910dfb3026\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:28.611096 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.611069    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11bc8568-9158-46b4-a967-63910dfb3026-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m\" (UID: \"11bc8568-9158-46b4-a967-63910dfb3026\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:28.668121 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.668086    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gmjw\" (UniqueName: \"kubernetes.io/projected/11bc8568-9158-46b4-a967-63910dfb3026-kube-api-access-4gmjw\") pod \"payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m\" (UID: \"11bc8568-9158-46b4-a967-63910dfb3026\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:28.723550 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.723513    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:28.941766 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:28.941740    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"]
Oct 30 22:09:28.944390 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:09:28.944355    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5bd9f42_ccb2_4bc4_b227_9abf27ac7b92.slice/crio-462d467902be5b31917c681634fdadf9c04e626135fea4d2680c7b20f25971d6 WatchSource:0}: Error finding container 462d467902be5b31917c681634fdadf9c04e626135fea4d2680c7b20f25971d6: Status 404 returned error can't find the container with id 462d467902be5b31917c681634fdadf9c04e626135fea4d2680c7b20f25971d6
Oct 30 22:09:29.100880 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.100851    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"]
Oct 30 22:09:29.104445 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:09:29.104418    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bc8568_9158_46b4_a967_63910dfb3026.slice/crio-850f21eb1ab87ad32aed08fe374866401a4c42b500deb8391f071d501685f3a5 WatchSource:0}: Error finding container 850f21eb1ab87ad32aed08fe374866401a4c42b500deb8391f071d501685f3a5: Status 404 returned error can't find the container with id 850f21eb1ab87ad32aed08fe374866401a4c42b500deb8391f071d501685f3a5
Oct 30 22:09:29.676840 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.676779    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m" event={"ID":"11bc8568-9158-46b4-a967-63910dfb3026","Type":"ContainerStarted","Data":"c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58"}
Oct 30 22:09:29.677073 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.676863    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m" event={"ID":"11bc8568-9158-46b4-a967-63910dfb3026","Type":"ContainerStarted","Data":"850f21eb1ab87ad32aed08fe374866401a4c42b500deb8391f071d501685f3a5"}
Oct 30 22:09:29.677073 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.676897    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:29.678359 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.678336    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw" event={"ID":"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92","Type":"ContainerStarted","Data":"a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5"}
Oct 30 22:09:29.678470 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.678379    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw" event={"ID":"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92","Type":"ContainerStarted","Data":"462d467902be5b31917c681634fdadf9c04e626135fea4d2680c7b20f25971d6"}
Oct 30 22:09:29.678470 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.678415    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:29.791834 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.791776    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m" podStartSLOduration=1.791760589 podStartE2EDuration="1.791760589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:29.789656581 +0000 UTC m=+3425.660775700" watchObservedRunningTime="2026-10-30 22:09:29.791760589 +0000 UTC m=+3425.662879706"
Oct 30 22:09:29.917717 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:29.916818    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw" podStartSLOduration=1.9167979819999998 podStartE2EDuration="1.916797982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:29.914405871 +0000 UTC m=+3425.785524993" watchObservedRunningTime="2026-10-30 22:09:29.916797982 +0000 UTC m=+3425.787917103"
Oct 30 22:09:40.686871 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:40.686843    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:09:40.687428 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:40.686907    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:09:48.824886 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:48.824852    5786 generic.go:356] "Generic (PLEG): container finished" podID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerID="7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333" exitCode=0
Oct 30 22:09:48.825490 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:48.824931    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" event={"ID":"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b","Type":"ContainerDied","Data":"7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333"}
Oct 30 22:09:48.825490 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:48.825464    5786 scope.go:122] "RemoveContainer" containerID="7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333"
Oct 30 22:09:48.826745 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:48.826717    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:09:48.826809 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:48.826788    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:09:48.826846 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:48.826817    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:09:49.831928 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:49.831881    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" event={"ID":"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b","Type":"ContainerStarted","Data":"34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d"}
Oct 30 22:09:50.473234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:50.470083    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:50.477179 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:50.477145    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused" start-of-body=
Oct 30 22:09:50.477369 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:50.477242    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused"
Oct 30 22:09:51.466234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:51.459671    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused" start-of-body=
Oct 30 22:09:51.466234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:51.459735    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused"
Oct 30 22:09:52.380562 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:09:52.380523    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:09:52.467234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:52.466621    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused" start-of-body=
Oct 30 22:09:52.467234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:52.466686    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused"
Oct 30 22:09:52.631663 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:52.631577    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:53.471234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:53.466676    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused" start-of-body=
Oct 30 22:09:53.471234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:53.466753    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused"
Oct 30 22:09:54.476372 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:54.476330    5786 patch_prober.go:29] interesting pod/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused" start-of-body=
Oct 30 22:09:54.476806 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:54.476415    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.244:15021/healthz/ready\": dial tcp 10.134.0.244:15021: connect: connection refused"
Oct 30 22:09:55.462959 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:55.462925    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:09:55.464103 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:09:55.464081    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:10:07.049459 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:07.049421    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"]
Oct 30 22:10:07.098576 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:07.098542    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2ebebd5-vfvr7"]
Oct 30 22:10:08.358929 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:08.358897    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="209f5887-36b8-48c6-8427-09a2376ddca7" path="/var/lib/kubelet/pods/209f5887-36b8-48c6-8427-09a2376ddca7/volumes"
Oct 30 22:10:22.354898 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:10:22.354857    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:10:52.356155 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:10:52.356119    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:10:54.347570 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.347536    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"]
Oct 30 22:10:54.351218 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.351186    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.405222 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.405183    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4bt6\" (UniqueName: \"kubernetes.io/projected/63232033-08b4-4b7d-a00b-3988df1d9077-kube-api-access-l4bt6\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.405385 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.405300    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.405433 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.405409    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.452896 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.452855    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:10:54.452896 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.452884    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:10:54.453127 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.452894    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:10:54.453127 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.452969    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:10:54.454751 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.454732    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"]
Oct 30 22:10:54.506763 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.506736    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4bt6\" (UniqueName: \"kubernetes.io/projected/63232033-08b4-4b7d-a00b-3988df1d9077-kube-api-access-l4bt6\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.506939 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.506793    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.506939 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.506854    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.507502 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.507478    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.509330 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.509313    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.562722 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.562696    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4bt6\" (UniqueName: \"kubernetes.io/projected/63232033-08b4-4b7d-a00b-3988df1d9077-kube-api-access-l4bt6\") pod \"maas-api-revoke-sub-bef2f275-b8hpm\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:54.664807 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:54.664785    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:55.036717 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:55.036694    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"]
Oct 30 22:10:55.039865 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:10:55.039837    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63232033_08b4_4b7d_a00b_3988df1d9077.slice/crio-e67c4c7b922cdd06711f96c64414f8d04ede6d8a6188f9d01ee7a8358d09e118 WatchSource:0}: Error finding container e67c4c7b922cdd06711f96c64414f8d04ede6d8a6188f9d01ee7a8358d09e118: Status 404 returned error can't find the container with id e67c4c7b922cdd06711f96c64414f8d04ede6d8a6188f9d01ee7a8358d09e118
Oct 30 22:10:55.193393 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:55.193356    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm" event={"ID":"63232033-08b4-4b7d-a00b-3988df1d9077","Type":"ContainerStarted","Data":"e67c4c7b922cdd06711f96c64414f8d04ede6d8a6188f9d01ee7a8358d09e118"}
Oct 30 22:10:56.200590 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:56.200553    5786 generic.go:356] "Generic (PLEG): container finished" podID="63232033-08b4-4b7d-a00b-3988df1d9077" containerID="3722d94db258cf2920fda1b1984df33202a67de358a5a56c4b3c1a37e8734429" exitCode=0
Oct 30 22:10:56.201021 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:56.200642    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm" event={"ID":"63232033-08b4-4b7d-a00b-3988df1d9077","Type":"ContainerDied","Data":"3722d94db258cf2920fda1b1984df33202a67de358a5a56c4b3c1a37e8734429"}
Oct 30 22:10:57.346394 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.346371    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:57.437022 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.436994    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63232033-08b4-4b7d-a00b-3988df1d9077-kube-api-access-l4bt6\" (UniqueName: \"kubernetes.io/projected/63232033-08b4-4b7d-a00b-3988df1d9077-kube-api-access-l4bt6\") pod \"63232033-08b4-4b7d-a00b-3988df1d9077\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") "
Oct 30 22:10:57.437185 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.437035    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-openshift-ca\") pod \"63232033-08b4-4b7d-a00b-3988df1d9077\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") "
Oct 30 22:10:57.437185 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.437121    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-xks-ca\") pod \"63232033-08b4-4b7d-a00b-3988df1d9077\" (UID: \"63232033-08b4-4b7d-a00b-3988df1d9077\") "
Oct 30 22:10:57.437447 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.437426    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-openshift-ca" pod "63232033-08b4-4b7d-a00b-3988df1d9077" (UID: "63232033-08b4-4b7d-a00b-3988df1d9077"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:57.437603 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.437588    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.439374 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.439344    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63232033-08b4-4b7d-a00b-3988df1d9077-kube-api-access-l4bt6" pod "63232033-08b4-4b7d-a00b-3988df1d9077" (UID: "63232033-08b4-4b7d-a00b-3988df1d9077"). InnerVolumeSpecName "kube-api-access-l4bt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:57.439470 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.439390    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-xks-ca" pod "63232033-08b4-4b7d-a00b-3988df1d9077" (UID: "63232033-08b4-4b7d-a00b-3988df1d9077"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:57.539042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.538976    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4bt6\" (UniqueName: \"kubernetes.io/projected/63232033-08b4-4b7d-a00b-3988df1d9077-kube-api-access-l4bt6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.539042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:57.539002    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63232033-08b4-4b7d-a00b-3988df1d9077-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:58.211803 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:58.211774    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"
Oct 30 22:10:58.211989 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:58.211772    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm" event={"ID":"63232033-08b4-4b7d-a00b-3988df1d9077","Type":"ContainerDied","Data":"e67c4c7b922cdd06711f96c64414f8d04ede6d8a6188f9d01ee7a8358d09e118"}
Oct 30 22:10:58.211989 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:10:58.211898    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e67c4c7b922cdd06711f96c64414f8d04ede6d8a6188f9d01ee7a8358d09e118"
Oct 30 22:11:05.913839 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:05.913754    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"]
Oct 30 22:11:05.914247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:05.914226    5786 cpu_manager.go:414] "containerMap: removing container" podUID="63232033-08b4-4b7d-a00b-3988df1d9077" containerName="revoke-keys"
Oct 30 22:11:05.914247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:05.914238    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63232033-08b4-4b7d-a00b-3988df1d9077" containerName="revoke-keys"
Oct 30 22:11:05.914348 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:05.914338    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="63232033-08b4-4b7d-a00b-3988df1d9077" containerName="revoke-keys"
Oct 30 22:11:05.917592 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:05.917577    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:05.972105 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:05.972061    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:11:06.021473 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.021442    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:11:06.021651 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.021540    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:11:06.021651 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.021535    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:11:06.023397 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.023375    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"]
Oct 30 22:11:06.024978 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.024953    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drn9r\" (UniqueName: \"kubernetes.io/projected/311e1538-805e-41ef-85c7-c47aa14dbcdf-kube-api-access-drn9r\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.025078 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.025043    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.025078 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.025074    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.125841 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.125804    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.126020 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.125848    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.126070 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.126030    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drn9r\" (UniqueName: \"kubernetes.io/projected/311e1538-805e-41ef-85c7-c47aa14dbcdf-kube-api-access-drn9r\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.126651 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.126576    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.128179 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.128156    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.183657 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.183587    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drn9r\" (UniqueName: \"kubernetes.io/projected/311e1538-805e-41ef-85c7-c47aa14dbcdf-kube-api-access-drn9r\") pod \"maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.230173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.230129    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:06.603335 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:06.603306    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"]
Oct 30 22:11:06.606362 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:11:06.606338    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311e1538_805e_41ef_85c7_c47aa14dbcdf.slice/crio-db9bd15c1f5a671b53cb9fcc2bf7acdfc7a0ff606ec27b2b46de5fbc20ae8832 WatchSource:0}: Error finding container db9bd15c1f5a671b53cb9fcc2bf7acdfc7a0ff606ec27b2b46de5fbc20ae8832: Status 404 returned error can't find the container with id db9bd15c1f5a671b53cb9fcc2bf7acdfc7a0ff606ec27b2b46de5fbc20ae8832
Oct 30 22:11:07.259499 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:07.259464    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2" event={"ID":"311e1538-805e-41ef-85c7-c47aa14dbcdf","Type":"ContainerStarted","Data":"aed01c86d34b90ebf95e71864e574ffa50d1f6254f34d65a44f495e1ce8f5f3b"}
Oct 30 22:11:07.259908 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:07.259519    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2" event={"ID":"311e1538-805e-41ef-85c7-c47aa14dbcdf","Type":"ContainerStarted","Data":"db9bd15c1f5a671b53cb9fcc2bf7acdfc7a0ff606ec27b2b46de5fbc20ae8832"}
Oct 30 22:11:07.377437 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:07.377388    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2" podStartSLOduration=2.377374027 podStartE2EDuration="2.377374027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:07.375922789 +0000 UTC m=+3523.247041910" watchObservedRunningTime="2026-10-30 22:11:07.377374027 +0000 UTC m=+3523.248493146"
Oct 30 22:11:08.047982 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:08.047950    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"]
Oct 30 22:11:08.097310 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:08.097280    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-schnz"]
Oct 30 22:11:08.267401 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:08.267360    5786 generic.go:356] "Generic (PLEG): container finished" podID="311e1538-805e-41ef-85c7-c47aa14dbcdf" containerID="aed01c86d34b90ebf95e71864e574ffa50d1f6254f34d65a44f495e1ce8f5f3b" exitCode=0
Oct 30 22:11:08.267401 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:08.267398    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2" event={"ID":"311e1538-805e-41ef-85c7-c47aa14dbcdf","Type":"ContainerDied","Data":"aed01c86d34b90ebf95e71864e574ffa50d1f6254f34d65a44f495e1ce8f5f3b"}
Oct 30 22:11:08.376276 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:08.375844    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f2b948c-d2dc-4750-87f4-33f9c0570a44" path="/var/lib/kubelet/pods/6f2b948c-d2dc-4750-87f4-33f9c0570a44/volumes"
Oct 30 22:11:09.409188 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.409165    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:09.584584 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.584496    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-xks-ca\") pod \"311e1538-805e-41ef-85c7-c47aa14dbcdf\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") "
Oct 30 22:11:09.584767 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.584698    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/311e1538-805e-41ef-85c7-c47aa14dbcdf-kube-api-access-drn9r\" (UniqueName: \"kubernetes.io/projected/311e1538-805e-41ef-85c7-c47aa14dbcdf-kube-api-access-drn9r\") pod \"311e1538-805e-41ef-85c7-c47aa14dbcdf\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") "
Oct 30 22:11:09.584767 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.584749    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-openshift-ca\") pod \"311e1538-805e-41ef-85c7-c47aa14dbcdf\" (UID: \"311e1538-805e-41ef-85c7-c47aa14dbcdf\") "
Oct 30 22:11:09.585152 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.585127    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-openshift-ca" pod "311e1538-805e-41ef-85c7-c47aa14dbcdf" (UID: "311e1538-805e-41ef-85c7-c47aa14dbcdf"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:09.586776 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.586750    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-xks-ca" pod "311e1538-805e-41ef-85c7-c47aa14dbcdf" (UID: "311e1538-805e-41ef-85c7-c47aa14dbcdf"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:09.587001 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.586976    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311e1538-805e-41ef-85c7-c47aa14dbcdf-kube-api-access-drn9r" pod "311e1538-805e-41ef-85c7-c47aa14dbcdf" (UID: "311e1538-805e-41ef-85c7-c47aa14dbcdf"). InnerVolumeSpecName "kube-api-access-drn9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:09.685902 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.685866    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drn9r\" (UniqueName: \"kubernetes.io/projected/311e1538-805e-41ef-85c7-c47aa14dbcdf-kube-api-access-drn9r\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:09.685902 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.685896    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:09.686105 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:09.685910    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/311e1538-805e-41ef-85c7-c47aa14dbcdf-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.279120 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:10.279090    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"
Oct 30 22:11:10.279120 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:10.279104    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2" event={"ID":"311e1538-805e-41ef-85c7-c47aa14dbcdf","Type":"ContainerDied","Data":"db9bd15c1f5a671b53cb9fcc2bf7acdfc7a0ff606ec27b2b46de5fbc20ae8832"}
Oct 30 22:11:10.279366 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:10.279180    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db9bd15c1f5a671b53cb9fcc2bf7acdfc7a0ff606ec27b2b46de5fbc20ae8832"
Oct 30 22:11:17.415097 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:17.415065    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"]
Oct 30 22:11:17.517827 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:17.517793    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"]
Oct 30 22:11:22.373237 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:11:22.372025    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:11:22.375630 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:22.375594    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"]
Oct 30 22:11:22.416322 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:22.416279    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m" podUID="11bc8568-9158-46b4-a967-63910dfb3026" containerName="payload-processing" containerID="cri-o://c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58" gracePeriod=25
Oct 30 22:11:22.421639 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:22.421611    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-dc7946-qgzf2"]
Oct 30 22:11:22.518790 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:22.518746    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw" podUID="a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" containerName="payload-pre-processing" containerID="cri-o://a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5" gracePeriod=25
Oct 30 22:11:24.359388 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:24.359355    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="311e1538-805e-41ef-85c7-c47aa14dbcdf" path="/var/lib/kubelet/pods/311e1538-805e-41ef-85c7-c47aa14dbcdf/volumes"
Oct 30 22:11:28.098262 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:28.098231    5786 scope.go:122] "RemoveContainer" containerID="e40c3e9fb0e64d9727fd86f5c121084cf22538cc3653e08b47d1cbc478d06ca7"
Oct 30 22:11:28.107185 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:28.107164    5786 scope.go:122] "RemoveContainer" containerID="92a90defae65244638ba5d831ab59cf4b5dd710a358224c45fdc7375e6633676"
Oct 30 22:11:29.659917 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:29.659882    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"]
Oct 30 22:11:29.660494 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:29.660082    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy" containerID="cri-o://34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d" gracePeriod=30
Oct 30 22:11:30.685035 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:30.684991    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw" podUID="a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:11:30.685541 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:30.685250    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m" podUID="11bc8568-9158-46b4-a967-63910dfb3026" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:11:35.312276 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.312250    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:11:35.418423 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.418390    5786 generic.go:356] "Generic (PLEG): container finished" podID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerID="34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d" exitCode=0
Oct 30 22:11:35.418612 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.418438    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"
Oct 30 22:11:35.418612 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.418447    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" event={"ID":"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b","Type":"ContainerDied","Data":"34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d"}
Oct 30 22:11:35.418612 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.418488    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6" event={"ID":"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b","Type":"ContainerDied","Data":"f897d230bd40374a43f7c3c8860b1b2786e3036d585b2906c0b02cb78adea19d"}
Oct 30 22:11:35.418612 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.418510    5786 scope.go:122] "RemoveContainer" containerID="34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d"
Oct 30 22:11:35.427417 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.427397    5786 scope.go:122] "RemoveContainer" containerID="7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333"
Oct 30 22:11:35.435765 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.435740    5786 scope.go:122] "RemoveContainer" containerID="34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d"
Oct 30 22:11:35.436040 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:11:35.436020    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d\": container with ID starting with 34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d not found: ID does not exist" containerID="34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d"
Oct 30 22:11:35.436094 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.436050    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d"} err="failed to get container status \"34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d\": rpc error: code = NotFound desc = could not find container \"34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d\": container with ID starting with 34da4bbe31b05a6c58ea78e3b6a9b54256738e1d1fe67e19d743dd1c2364a19d not found: ID does not exist"
Oct 30 22:11:35.436094 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.436069    5786 scope.go:122] "RemoveContainer" containerID="7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333"
Oct 30 22:11:35.436322 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:11:35.436306    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333\": container with ID starting with 7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333 not found: ID does not exist" containerID="7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333"
Oct 30 22:11:35.436381 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.436327    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333"} err="failed to get container status \"7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333\": rpc error: code = NotFound desc = could not find container \"7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333\": container with ID starting with 7adb01f4f066c43a43a7510a741c64246695b2c6b7e4a1fe45e498236ee1a333 not found: ID does not exist"
Oct 30 22:11:35.465339 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465311    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-token\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-token\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.465485 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465450    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-podinfo\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.465527 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465500    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-data\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.465572 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465529    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-credential-socket\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.465572 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465563    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-certs\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.465785 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465749    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-kube-api-access-tl72p\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-kube-api-access-tl72p\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.465874 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465798    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-certs" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:35.465874 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465810    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-envoy\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.465990 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465875    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istiod-ca-cert\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.465990 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465809    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-data" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:35.465990 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465819    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-credential-socket" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:35.465990 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.465917    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-socket\") pod \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\" (UID: \"f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b\") "
Oct 30 22:11:35.466247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.466224    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-socket" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:35.466330 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.466269    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istiod-ca-cert" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:35.466384 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.466358    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.466445 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.466380    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.466445 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.466395    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.466445 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.466409    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.466445 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.466423    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.468243 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.468191    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-token" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:35.468366 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.468286    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-podinfo" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:35.468411 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.468373    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-envoy" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:35.468411 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.468392    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-kube-api-access-tl72p" pod "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" (UID: "f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b"). InnerVolumeSpecName "kube-api-access-tl72p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:35.567245 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.567188    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.567245 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.567243    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.567473 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.567262    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl72p\" (UniqueName: \"kubernetes.io/projected/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-kube-api-access-tl72p\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.567473 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.567279    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:35.837616 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.837584    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"]
Oct 30 22:11:35.839124 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:35.839103    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-dc7946-openshift-default-cb8dd685c-vvgd6"]
Oct 30 22:11:36.361891 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:36.361859    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" path="/var/lib/kubelet/pods/f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b/volumes"
Oct 30 22:11:40.684881 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:40.684836    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m" podUID="11bc8568-9158-46b4-a967-63910dfb3026" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:11:40.685329 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:40.684851    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw" podUID="a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:11:43.665021 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.546079    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:11:43.675253 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.666817    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11bc8568-9158-46b4-a967-63910dfb3026-kube-api-access-4gmjw\" (UniqueName: \"kubernetes.io/projected/11bc8568-9158-46b4-a967-63910dfb3026-kube-api-access-4gmjw\") pod \"11bc8568-9158-46b4-a967-63910dfb3026\" (UID: \"11bc8568-9158-46b4-a967-63910dfb3026\") "
Oct 30 22:11:43.675253 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.674805    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11bc8568-9158-46b4-a967-63910dfb3026-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11bc8568-9158-46b4-a967-63910dfb3026-plugins-config-volume\") pod \"11bc8568-9158-46b4-a967-63910dfb3026\" (UID: \"11bc8568-9158-46b4-a967-63910dfb3026\") "
Oct 30 22:11:43.687369 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.684425    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11bc8568-9158-46b4-a967-63910dfb3026-plugins-config-volume" pod "11bc8568-9158-46b4-a967-63910dfb3026" (UID: "11bc8568-9158-46b4-a967-63910dfb3026"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:43.759233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.733309    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-zr2km container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:11:43.759233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.733374    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:11:43.788046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.788006    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11bc8568-9158-46b4-a967-63910dfb3026-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.926233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.925760    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"
Oct 30 22:11:43.926966 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.926480    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m" event={"ID":"11bc8568-9158-46b4-a967-63910dfb3026","Type":"ContainerDied","Data":"c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58"}
Oct 30 22:11:43.926966 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.926563    5786 scope.go:122] "RemoveContainer" containerID="c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58"
Oct 30 22:11:43.927635 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.927601    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:11:43.929029 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.929007    5786 generic.go:356] "Generic (PLEG): container finished" podID="11bc8568-9158-46b4-a967-63910dfb3026" containerID="c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58" exitCode=0
Oct 30 22:11:43.929394 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.929319    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m" event={"ID":"11bc8568-9158-46b4-a967-63910dfb3026","Type":"ContainerDied","Data":"850f21eb1ab87ad32aed08fe374866401a4c42b500deb8391f071d501685f3a5"}
Oct 30 22:11:43.935958 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.935589    5786 generic.go:356] "Generic (PLEG): container finished" podID="a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" containerID="a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5" exitCode=0
Oct 30 22:11:43.935958 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.935689    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw" event={"ID":"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92","Type":"ContainerDied","Data":"a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5"}
Oct 30 22:11:43.935958 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.935748    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw" event={"ID":"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92","Type":"ContainerDied","Data":"462d467902be5b31917c681634fdadf9c04e626135fea4d2680c7b20f25971d6"}
Oct 30 22:11:43.935958 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.935819    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"
Oct 30 22:11:43.957087 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:43.957013    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11bc8568-9158-46b4-a967-63910dfb3026-kube-api-access-4gmjw" pod "11bc8568-9158-46b4-a967-63910dfb3026" (UID: "11bc8568-9158-46b4-a967-63910dfb3026"). InnerVolumeSpecName "kube-api-access-4gmjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:44.022550 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.022458    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-plugins-config-volume\") pod \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\" (UID: \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\") "
Oct 30 22:11:44.038268 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.023257    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-kube-api-access-jnsj6\" (UniqueName: \"kubernetes.io/projected/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-kube-api-access-jnsj6\") pod \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\" (UID: \"a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92\") "
Oct 30 22:11:44.038268 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.023666    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gmjw\" (UniqueName: \"kubernetes.io/projected/11bc8568-9158-46b4-a967-63910dfb3026-kube-api-access-4gmjw\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:44.040360 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.039957    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-plugins-config-volume" pod "a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" (UID: "a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:44.042590 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.042483    5786 scope.go:122] "RemoveContainer" containerID="c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58"
Oct 30 22:11:44.054409 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:11:44.044635    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58\": container with ID starting with c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58 not found: ID does not exist" containerID="c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58"
Oct 30 22:11:44.054409 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.044698    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58"} err="failed to get container status \"c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58\": rpc error: code = NotFound desc = could not find container \"c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58\": container with ID starting with c6076acff138f49651037abf2c3198e7d598098d7e45dd31216f3e2611f34c58 not found: ID does not exist"
Oct 30 22:11:44.054409 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.044723    5786 scope.go:122] "RemoveContainer" containerID="a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5"
Oct 30 22:11:44.070691 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.070638    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-kube-api-access-jnsj6" pod "a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" (UID: "a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92"). InnerVolumeSpecName "kube-api-access-jnsj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:44.144871 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.144615    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-xtfjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.218:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:11:44.144871 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.144734    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.218:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:11:44.185239 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.184406    5786 scope.go:122] "RemoveContainer" containerID="a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5"
Oct 30 22:11:44.185239 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.184563    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnsj6\" (UniqueName: \"kubernetes.io/projected/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-kube-api-access-jnsj6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:44.185239 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.184602    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:44.185996 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:11:44.185942    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5\": container with ID starting with a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5 not found: ID does not exist" containerID="a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5"
Oct 30 22:11:44.186225 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.186161    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5"} err="failed to get container status \"a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5\": rpc error: code = NotFound desc = could not find container \"a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5\": container with ID starting with a3ee01cf1fa4df1f947dfefee3b922aa944d80860dc9f405e136b82ec134fda5 not found: ID does not exist"
Oct 30 22:11:44.404267 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.404232    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"]
Oct 30 22:11:44.412074 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.412043    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-dc7946-6f7fcc9ddztj7m"]
Oct 30 22:11:44.521256 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.521163    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"]
Oct 30 22:11:44.525617 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:44.525587    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-dc7946-7bfd4l2vtw"]
Oct 30 22:11:46.357921 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:46.357889    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11bc8568-9158-46b4-a967-63910dfb3026" path="/var/lib/kubelet/pods/11bc8568-9158-46b4-a967-63910dfb3026/volumes"
Oct 30 22:11:46.358326 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:11:46.358298    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" path="/var/lib/kubelet/pods/a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92/volumes"
Oct 30 22:11:52.355044 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:11:52.355009    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:12:18.448369 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448332    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"]
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448743    5786 cpu_manager.go:414] "containerMap: removing container" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy"
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448754    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy"
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448780    5786 cpu_manager.go:414] "containerMap: removing container" podUID="11bc8568-9158-46b4-a967-63910dfb3026" containerName="payload-processing"
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448786    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11bc8568-9158-46b4-a967-63910dfb3026" containerName="payload-processing"
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448797    5786 cpu_manager.go:414] "containerMap: removing container" podUID="311e1538-805e-41ef-85c7-c47aa14dbcdf" containerName="revoke-keys"
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448802    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="311e1538-805e-41ef-85c7-c47aa14dbcdf" containerName="revoke-keys"
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448817    5786 cpu_manager.go:414] "containerMap: removing container" podUID="a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" containerName="payload-pre-processing"
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448822    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" containerName="payload-pre-processing"
Oct 30 22:12:18.448885 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448885    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5bd9f42-ccb2-4bc4-b227-9abf27ac7b92" containerName="payload-pre-processing"
Oct 30 22:12:18.449268 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448893    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy"
Oct 30 22:12:18.449268 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448901    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy"
Oct 30 22:12:18.449268 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448912    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="11bc8568-9158-46b4-a967-63910dfb3026" containerName="payload-processing"
Oct 30 22:12:18.449268 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.448920    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="311e1538-805e-41ef-85c7-c47aa14dbcdf" containerName="revoke-keys"
Oct 30 22:12:18.452085 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.452070    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.536158 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.536121    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqkz8\" (UniqueName: \"kubernetes.io/projected/2cd60347-c431-4105-8e2f-7ec0324db9a7-kube-api-access-nqkz8\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.536352 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.536163    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-home\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.536352 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.536181    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-dshm\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.536436 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.536377    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd60347-c431-4105-8e2f-7ec0324db9a7-tls-certs\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.536436 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.536413    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-tmp-dir\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.536504 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.536437    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-model-cache\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.555163 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.555132    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-fb2c6f18-kserve-self-signed-certs\""
Oct 30 22:12:18.557025 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.556996    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"]
Oct 30 22:12:18.637721 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.637685    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd60347-c431-4105-8e2f-7ec0324db9a7-tls-certs\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.637933 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.637734    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-tmp-dir\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.637933 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.637768    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-model-cache\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.637933 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.637825    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqkz8\" (UniqueName: \"kubernetes.io/projected/2cd60347-c431-4105-8e2f-7ec0324db9a7-kube-api-access-nqkz8\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.637933 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.637857    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-home\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.637933 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.637880    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-dshm\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.638323 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.638284    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-tmp-dir\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.638323 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.638320    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-model-cache\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.638554 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.638394    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-home\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.640192 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.640170    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-dshm\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.640521 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.640504    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd60347-c431-4105-8e2f-7ec0324db9a7-tls-certs\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.695702 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.695671    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqkz8\" (UniqueName: \"kubernetes.io/projected/2cd60347-c431-4105-8e2f-7ec0324db9a7-kube-api-access-nqkz8\") pod \"e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") " pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:18.770453 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:18.770362    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:19.168735 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:19.168706    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"]
Oct 30 22:12:19.171250 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:12:19.171223    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cd60347_c431_4105_8e2f_7ec0324db9a7.slice/crio-4bf6928375ceb6052726f11efd3e22647a609942df8907a2cb1d31d530076e19 WatchSource:0}: Error finding container 4bf6928375ceb6052726f11efd3e22647a609942df8907a2cb1d31d530076e19: Status 404 returned error can't find the container with id 4bf6928375ceb6052726f11efd3e22647a609942df8907a2cb1d31d530076e19
Oct 30 22:12:20.115191 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:20.115105    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h" event={"ID":"2cd60347-c431-4105-8e2f-7ec0324db9a7","Type":"ContainerStarted","Data":"f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c"}
Oct 30 22:12:20.115191 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:20.115148    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h" event={"ID":"2cd60347-c431-4105-8e2f-7ec0324db9a7","Type":"ContainerStarted","Data":"4bf6928375ceb6052726f11efd3e22647a609942df8907a2cb1d31d530076e19"}
Oct 30 22:12:20.231737 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:20.231682    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h" podStartSLOduration=2.231664867 podStartE2EDuration="2.231664867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:20.231118687 +0000 UTC m=+3596.102237811" watchObservedRunningTime="2026-10-30 22:12:20.231664867 +0000 UTC m=+3596.102783986"
Oct 30 22:12:20.770687 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:20.770635    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:22.354889 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:12:22.354858    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:12:28.140481 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:28.140451    5786 scope.go:122] "RemoveContainer" containerID="10094d68e39f3bc6c80b63f78a6f0bda26e367fd352ae518e436668c58da6904"
Oct 30 22:12:28.771350 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:28.771308    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:28.783638 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:28.783618    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:29.210723 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:29.210693    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:12:52.354859 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:12:52.354825    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:12:52.695619 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.695576    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"]
Oct 30 22:12:52.696108 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.696088    5786 cpu_manager.go:414] "containerMap: removing container" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy"
Oct 30 22:12:52.696249 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.696110    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1ba3e4c-c596-4a2c-8193-ccb21c5f3e2b" containerName="istio-proxy"
Oct 30 22:12:52.699354 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.699333    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.756446 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.756407    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"]
Oct 30 22:12:52.803042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.803003    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:12:52.803042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.803029    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:12:52.803042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.803010    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:12:52.803042 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.803011    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:12:52.817562 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.817526    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.817712 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.817602    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndffn\" (UniqueName: \"kubernetes.io/projected/120df019-dde6-4b6b-b1c6-a44aa86746c8-kube-api-access-ndffn\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.817785 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.817763    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.918930 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.918890    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.919124 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.918947    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndffn\" (UniqueName: \"kubernetes.io/projected/120df019-dde6-4b6b-b1c6-a44aa86746c8-kube-api-access-ndffn\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.919124 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.919008    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.919760 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.919667    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.921468 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.921441    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:52.977049 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:52.976968    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndffn\" (UniqueName: \"kubernetes.io/projected/120df019-dde6-4b6b-b1c6-a44aa86746c8-kube-api-access-ndffn\") pod \"maas-api-revoke-sub-452b6d7b-dmnh6\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:53.012012 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:53.011980    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:53.390084 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:53.390059    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"]
Oct 30 22:12:53.392667 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:12:53.392639    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120df019_dde6_4b6b_b1c6_a44aa86746c8.slice/crio-9bdd113d8dd9a60e5aacecc73b5d6760b26f07a690072a33fcaeafcca0f2696b WatchSource:0}: Error finding container 9bdd113d8dd9a60e5aacecc73b5d6760b26f07a690072a33fcaeafcca0f2696b: Status 404 returned error can't find the container with id 9bdd113d8dd9a60e5aacecc73b5d6760b26f07a690072a33fcaeafcca0f2696b
Oct 30 22:12:54.325111 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:54.325077    5786 generic.go:356] "Generic (PLEG): container finished" podID="120df019-dde6-4b6b-b1c6-a44aa86746c8" containerID="7b98bf8bd0d390793c7b3e9fdfc3e9d8d05c801b9e225b95f076f068c2beb6a6" exitCode=0
Oct 30 22:12:54.325312 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:54.325166    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6" event={"ID":"120df019-dde6-4b6b-b1c6-a44aa86746c8","Type":"ContainerDied","Data":"7b98bf8bd0d390793c7b3e9fdfc3e9d8d05c801b9e225b95f076f068c2beb6a6"}
Oct 30 22:12:54.325312 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:54.325275    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6" event={"ID":"120df019-dde6-4b6b-b1c6-a44aa86746c8","Type":"ContainerStarted","Data":"9bdd113d8dd9a60e5aacecc73b5d6760b26f07a690072a33fcaeafcca0f2696b"}
Oct 30 22:12:55.530102 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.530059    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:55.658195 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.654543    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-xks-ca\") pod \"120df019-dde6-4b6b-b1c6-a44aa86746c8\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") "
Oct 30 22:12:55.658195 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.654668    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/120df019-dde6-4b6b-b1c6-a44aa86746c8-kube-api-access-ndffn\" (UniqueName: \"kubernetes.io/projected/120df019-dde6-4b6b-b1c6-a44aa86746c8-kube-api-access-ndffn\") pod \"120df019-dde6-4b6b-b1c6-a44aa86746c8\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") "
Oct 30 22:12:55.658195 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.654822    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-openshift-ca\") pod \"120df019-dde6-4b6b-b1c6-a44aa86746c8\" (UID: \"120df019-dde6-4b6b-b1c6-a44aa86746c8\") "
Oct 30 22:12:55.658195 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.655374    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-openshift-ca" pod "120df019-dde6-4b6b-b1c6-a44aa86746c8" (UID: "120df019-dde6-4b6b-b1c6-a44aa86746c8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:55.658195 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.658005    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/120df019-dde6-4b6b-b1c6-a44aa86746c8-kube-api-access-ndffn" pod "120df019-dde6-4b6b-b1c6-a44aa86746c8" (UID: "120df019-dde6-4b6b-b1c6-a44aa86746c8"). InnerVolumeSpecName "kube-api-access-ndffn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:55.658195 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.658075    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-xks-ca" pod "120df019-dde6-4b6b-b1c6-a44aa86746c8" (UID: "120df019-dde6-4b6b-b1c6-a44aa86746c8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:12:55.761235 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.755971    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:55.761235 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.756011    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/120df019-dde6-4b6b-b1c6-a44aa86746c8-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:55.761235 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:55.756031    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndffn\" (UniqueName: \"kubernetes.io/projected/120df019-dde6-4b6b-b1c6-a44aa86746c8-kube-api-access-ndffn\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:56.371609 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:56.371436    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"
Oct 30 22:12:56.410772 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:56.410728    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6" event={"ID":"120df019-dde6-4b6b-b1c6-a44aa86746c8","Type":"ContainerDied","Data":"9bdd113d8dd9a60e5aacecc73b5d6760b26f07a690072a33fcaeafcca0f2696b"}
Oct 30 22:12:56.410913 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:12:56.410787    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bdd113d8dd9a60e5aacecc73b5d6760b26f07a690072a33fcaeafcca0f2696b"
Oct 30 22:13:22.077232 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:22.075232    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"]
Oct 30 22:13:22.461681 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:13:22.461636    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:13:26.231007 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.230971    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"]
Oct 30 22:13:26.231485 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.231434    5786 cpu_manager.go:414] "containerMap: removing container" podUID="120df019-dde6-4b6b-b1c6-a44aa86746c8" containerName="revoke-keys"
Oct 30 22:13:26.231485 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.231446    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="120df019-dde6-4b6b-b1c6-a44aa86746c8" containerName="revoke-keys"
Oct 30 22:13:26.231573 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.231518    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="120df019-dde6-4b6b-b1c6-a44aa86746c8" containerName="revoke-keys"
Oct 30 22:13:26.234696 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.234679    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.287433 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.287403    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:13:26.287618 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.287440    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:13:26.287618 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.287433    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:13:26.287618 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.287402    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:13:26.338464 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.338434    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"]
Oct 30 22:13:26.398616 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.398583    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.398796 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.398700    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2kjq\" (UniqueName: \"kubernetes.io/projected/23a9006c-c90d-482c-ae13-33d24b45d897-kube-api-access-z2kjq\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.398842 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.398806    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.500316 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.500201    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2kjq\" (UniqueName: \"kubernetes.io/projected/23a9006c-c90d-482c-ae13-33d24b45d897-kube-api-access-z2kjq\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.500316 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.500321    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.500542 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.500378    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.500994 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.500975    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.503067 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.503038    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.556488 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.556453    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2kjq\" (UniqueName: \"kubernetes.io/projected/23a9006c-c90d-482c-ae13-33d24b45d897-kube-api-access-z2kjq\") pod \"maas-api-revoke-sub-3e31a506-fcslq\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:26.847838 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:26.847751    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:27.224276 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:27.224249    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"]
Oct 30 22:13:27.226969 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:13:27.226943    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a9006c_c90d_482c_ae13_33d24b45d897.slice/crio-ace968306d75015d170e40cd2f252576a2875293ef814bf25a6273c5b3a6c09b WatchSource:0}: Error finding container ace968306d75015d170e40cd2f252576a2875293ef814bf25a6273c5b3a6c09b: Status 404 returned error can't find the container with id ace968306d75015d170e40cd2f252576a2875293ef814bf25a6273c5b3a6c09b
Oct 30 22:13:27.559249 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:27.559176    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq" event={"ID":"23a9006c-c90d-482c-ae13-33d24b45d897","Type":"ContainerStarted","Data":"ace968306d75015d170e40cd2f252576a2875293ef814bf25a6273c5b3a6c09b"}
Oct 30 22:13:28.569344 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:28.569234    5786 generic.go:356] "Generic (PLEG): container finished" podID="23a9006c-c90d-482c-ae13-33d24b45d897" containerID="936f5f1915df5d16273b58f4189935ebabc5653b4daed9baa595f66f76bec6ea" exitCode=0
Oct 30 22:13:28.569873 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:28.569334    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq" event={"ID":"23a9006c-c90d-482c-ae13-33d24b45d897","Type":"ContainerDied","Data":"936f5f1915df5d16273b58f4189935ebabc5653b4daed9baa595f66f76bec6ea"}
Oct 30 22:13:30.376503 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.376418    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:30.473231 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.472600    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23a9006c-c90d-482c-ae13-33d24b45d897-kube-api-access-z2kjq\" (UniqueName: \"kubernetes.io/projected/23a9006c-c90d-482c-ae13-33d24b45d897-kube-api-access-z2kjq\") pod \"23a9006c-c90d-482c-ae13-33d24b45d897\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") "
Oct 30 22:13:30.473231 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.472697    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-xks-ca\") pod \"23a9006c-c90d-482c-ae13-33d24b45d897\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") "
Oct 30 22:13:30.473231 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.472778    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-openshift-ca\") pod \"23a9006c-c90d-482c-ae13-33d24b45d897\" (UID: \"23a9006c-c90d-482c-ae13-33d24b45d897\") "
Oct 30 22:13:30.477415 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.477384    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-openshift-ca" pod "23a9006c-c90d-482c-ae13-33d24b45d897" (UID: "23a9006c-c90d-482c-ae13-33d24b45d897"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:30.536233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.515178    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23a9006c-c90d-482c-ae13-33d24b45d897-kube-api-access-z2kjq" pod "23a9006c-c90d-482c-ae13-33d24b45d897" (UID: "23a9006c-c90d-482c-ae13-33d24b45d897"). InnerVolumeSpecName "kube-api-access-z2kjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:30.550752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.550712    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-xks-ca" pod "23a9006c-c90d-482c-ae13-33d24b45d897" (UID: "23a9006c-c90d-482c-ae13-33d24b45d897"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:30.596321 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.596068    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2kjq\" (UniqueName: \"kubernetes.io/projected/23a9006c-c90d-482c-ae13-33d24b45d897-kube-api-access-z2kjq\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:30.596321 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.596111    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:30.596321 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.596129    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/23a9006c-c90d-482c-ae13-33d24b45d897-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:30.717247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.713834    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq" event={"ID":"23a9006c-c90d-482c-ae13-33d24b45d897","Type":"ContainerDied","Data":"ace968306d75015d170e40cd2f252576a2875293ef814bf25a6273c5b3a6c09b"}
Oct 30 22:13:30.717247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.713920    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ace968306d75015d170e40cd2f252576a2875293ef814bf25a6273c5b3a6c09b"
Oct 30 22:13:30.717247 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:30.714014    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"
Oct 30 22:13:32.049536 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:32.049496    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"]
Oct 30 22:13:32.099092 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:32.099058    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-75vm4"]
Oct 30 22:13:32.358313 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:32.358201    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80b0d54b-734d-4c2d-9b5d-5415054ce9c3" path="/var/lib/kubelet/pods/80b0d54b-734d-4c2d-9b5d-5415054ce9c3/volumes"
Oct 30 22:13:37.111170 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.111111    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h" podUID="2cd60347-c431-4105-8e2f-7ec0324db9a7" containerName="main" containerID="cri-o://f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c" gracePeriod=45
Oct 30 22:13:37.200308 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.200265    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h" podUID="2cd60347-c431-4105-8e2f-7ec0324db9a7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.249:8000/ready\": dial tcp 10.134.0.249:8000: connect: connection refused"
Oct 30 22:13:37.369081 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.369020    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:13:37.417269 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417238    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-home\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-home\") pod \"2cd60347-c431-4105-8e2f-7ec0324db9a7\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") "
Oct 30 22:13:37.417426 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417315    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-dshm\") pod \"2cd60347-c431-4105-8e2f-7ec0324db9a7\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") "
Oct 30 22:13:37.417505 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417436    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-tmp-dir\") pod \"2cd60347-c431-4105-8e2f-7ec0324db9a7\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") "
Oct 30 22:13:37.417505 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417449    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-home" pod "2cd60347-c431-4105-8e2f-7ec0324db9a7" (UID: "2cd60347-c431-4105-8e2f-7ec0324db9a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:37.417505 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417483    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2cd60347-c431-4105-8e2f-7ec0324db9a7-tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd60347-c431-4105-8e2f-7ec0324db9a7-tls-certs\") pod \"2cd60347-c431-4105-8e2f-7ec0324db9a7\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") "
Oct 30 22:13:37.417678 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417531    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-model-cache\") pod \"2cd60347-c431-4105-8e2f-7ec0324db9a7\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") "
Oct 30 22:13:37.417678 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417590    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cd60347-c431-4105-8e2f-7ec0324db9a7-kube-api-access-nqkz8\" (UniqueName: \"kubernetes.io/projected/2cd60347-c431-4105-8e2f-7ec0324db9a7-kube-api-access-nqkz8\") pod \"2cd60347-c431-4105-8e2f-7ec0324db9a7\" (UID: \"2cd60347-c431-4105-8e2f-7ec0324db9a7\") "
Oct 30 22:13:37.417759 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417684    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-tmp-dir" pod "2cd60347-c431-4105-8e2f-7ec0324db9a7" (UID: "2cd60347-c431-4105-8e2f-7ec0324db9a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:37.417817 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417799    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-model-cache" pod "2cd60347-c431-4105-8e2f-7ec0324db9a7" (UID: "2cd60347-c431-4105-8e2f-7ec0324db9a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:37.417949 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417934    5786 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-model-cache\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:37.418022 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417952    5786 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-home\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:37.418022 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.417962    5786 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-tmp-dir\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:37.419623 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.419601    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cd60347-c431-4105-8e2f-7ec0324db9a7-tls-certs" pod "2cd60347-c431-4105-8e2f-7ec0324db9a7" (UID: "2cd60347-c431-4105-8e2f-7ec0324db9a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:37.419738 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.419604    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-dshm" pod "2cd60347-c431-4105-8e2f-7ec0324db9a7" (UID: "2cd60347-c431-4105-8e2f-7ec0324db9a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:37.419901 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.419792    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cd60347-c431-4105-8e2f-7ec0324db9a7-kube-api-access-nqkz8" pod "2cd60347-c431-4105-8e2f-7ec0324db9a7" (UID: "2cd60347-c431-4105-8e2f-7ec0324db9a7"). InnerVolumeSpecName "kube-api-access-nqkz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:37.518849 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.518818    5786 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd60347-c431-4105-8e2f-7ec0324db9a7-dshm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:37.518849 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.518846    5786 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd60347-c431-4105-8e2f-7ec0324db9a7-tls-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:37.519043 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.518856    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqkz8\" (UniqueName: \"kubernetes.io/projected/2cd60347-c431-4105-8e2f-7ec0324db9a7-kube-api-access-nqkz8\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:37.750074 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.750042    5786 generic.go:356] "Generic (PLEG): container finished" podID="2cd60347-c431-4105-8e2f-7ec0324db9a7" containerID="f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c" exitCode=0
Oct 30 22:13:37.750288 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.750092    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"
Oct 30 22:13:37.750288 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.750106    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h" event={"ID":"2cd60347-c431-4105-8e2f-7ec0324db9a7","Type":"ContainerDied","Data":"f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c"}
Oct 30 22:13:37.750288 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.750144    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h" event={"ID":"2cd60347-c431-4105-8e2f-7ec0324db9a7","Type":"ContainerDied","Data":"4bf6928375ceb6052726f11efd3e22647a609942df8907a2cb1d31d530076e19"}
Oct 30 22:13:37.750288 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.750167    5786 scope.go:122] "RemoveContainer" containerID="f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c"
Oct 30 22:13:37.759768 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.759747    5786 scope.go:122] "RemoveContainer" containerID="f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c"
Oct 30 22:13:37.760024 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:13:37.760006    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c\": container with ID starting with f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c not found: ID does not exist" containerID="f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c"
Oct 30 22:13:37.760071 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.760033    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c"} err="failed to get container status \"f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c\": rpc error: code = NotFound desc = could not find container \"f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c\": container with ID starting with f48d2dd03f5ea645efaff97bc29c6e9bb206d2f0b9d0f89025cbe7415d00cd9c not found: ID does not exist"
Oct 30 22:13:37.915395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.915361    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"]
Oct 30 22:13:37.964651 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:37.964621    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-fb2c6f18-kserve-7766f9ddfc-gpr7h"]
Oct 30 22:13:38.358335 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:38.358302    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2cd60347-c431-4105-8e2f-7ec0324db9a7" path="/var/lib/kubelet/pods/2cd60347-c431-4105-8e2f-7ec0324db9a7/volumes"
Oct 30 22:13:52.354936 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:13:52.354900    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:13:55.646817 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.646782    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"]
Oct 30 22:13:55.647269 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.647249    5786 cpu_manager.go:414] "containerMap: removing container" podUID="23a9006c-c90d-482c-ae13-33d24b45d897" containerName="revoke-keys"
Oct 30 22:13:55.647269 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.647262    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23a9006c-c90d-482c-ae13-33d24b45d897" containerName="revoke-keys"
Oct 30 22:13:55.647363 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.647287    5786 cpu_manager.go:414] "containerMap: removing container" podUID="2cd60347-c431-4105-8e2f-7ec0324db9a7" containerName="main"
Oct 30 22:13:55.647363 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.647292    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cd60347-c431-4105-8e2f-7ec0324db9a7" containerName="main"
Oct 30 22:13:55.647433 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.647378    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="23a9006c-c90d-482c-ae13-33d24b45d897" containerName="revoke-keys"
Oct 30 22:13:55.647433 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.647392    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cd60347-c431-4105-8e2f-7ec0324db9a7" containerName="main"
Oct 30 22:13:55.650599 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.650578    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.704500 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.704456    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:13:55.707295 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.707265    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"]
Oct 30 22:13:55.753877 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.753849    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:13:55.753877 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.753860    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:13:55.754100 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.753926    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vmmz2\""
Oct 30 22:13:55.794508 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.794473    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnpqm\" (UniqueName: \"kubernetes.io/projected/5567fb4d-f425-4536-ba0c-c97271c15b92-kube-api-access-vnpqm\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.794695 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.794626    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.794695 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.794666    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.895848 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.895812    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnpqm\" (UniqueName: \"kubernetes.io/projected/5567fb4d-f425-4536-ba0c-c97271c15b92-kube-api-access-vnpqm\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.896031 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.895915    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.896031 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.895940    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.896608 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.896579    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.898511 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.898455    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.952466 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.952430    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnpqm\" (UniqueName: \"kubernetes.io/projected/5567fb4d-f425-4536-ba0c-c97271c15b92-kube-api-access-vnpqm\") pod \"maas-api-revoke-sub-ec5c9b19-wzv4d\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:55.963365 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:55.963339    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:56.340189 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:56.340160    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"]
Oct 30 22:13:56.342832 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:13:56.342802    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5567fb4d_f425_4536_ba0c_c97271c15b92.slice/crio-2932ffc516cef388d1822398d1247bbe61e714332b7dee5b7838833f4cc92ddd WatchSource:0}: Error finding container 2932ffc516cef388d1822398d1247bbe61e714332b7dee5b7838833f4cc92ddd: Status 404 returned error can't find the container with id 2932ffc516cef388d1822398d1247bbe61e714332b7dee5b7838833f4cc92ddd
Oct 30 22:13:56.924853 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:56.924821    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d" event={"ID":"5567fb4d-f425-4536-ba0c-c97271c15b92","Type":"ContainerStarted","Data":"542aefb0e826db80a4b8548122da8a98b7af2b8b55cf5686f5265afe614b317c"}
Oct 30 22:13:56.925280 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:56.924873    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d" event={"ID":"5567fb4d-f425-4536-ba0c-c97271c15b92","Type":"ContainerStarted","Data":"2932ffc516cef388d1822398d1247bbe61e714332b7dee5b7838833f4cc92ddd"}
Oct 30 22:13:57.039011 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:57.038945    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d" podStartSLOduration=2.03892964 podStartE2EDuration="2.03892964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:57.037774904 +0000 UTC m=+3692.908894025" watchObservedRunningTime="2026-10-30 22:13:57.03892964 +0000 UTC m=+3692.910048753"
Oct 30 22:13:57.931650 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:57.931621    5786 generic.go:356] "Generic (PLEG): container finished" podID="5567fb4d-f425-4536-ba0c-c97271c15b92" containerID="542aefb0e826db80a4b8548122da8a98b7af2b8b55cf5686f5265afe614b317c" exitCode=0
Oct 30 22:13:57.932064 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:57.931711    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d" event={"ID":"5567fb4d-f425-4536-ba0c-c97271c15b92","Type":"ContainerDied","Data":"542aefb0e826db80a4b8548122da8a98b7af2b8b55cf5686f5265afe614b317c"}
Oct 30 22:13:59.082911 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.082889    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:59.228995 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.228911    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-openshift-ca\") pod \"5567fb4d-f425-4536-ba0c-c97271c15b92\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") "
Oct 30 22:13:59.229153 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.229004    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-xks-ca\") pod \"5567fb4d-f425-4536-ba0c-c97271c15b92\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") "
Oct 30 22:13:59.229153 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.229125    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5567fb4d-f425-4536-ba0c-c97271c15b92-kube-api-access-vnpqm\" (UniqueName: \"kubernetes.io/projected/5567fb4d-f425-4536-ba0c-c97271c15b92-kube-api-access-vnpqm\") pod \"5567fb4d-f425-4536-ba0c-c97271c15b92\" (UID: \"5567fb4d-f425-4536-ba0c-c97271c15b92\") "
Oct 30 22:13:59.229304 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.229278    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-openshift-ca" pod "5567fb4d-f425-4536-ba0c-c97271c15b92" (UID: "5567fb4d-f425-4536-ba0c-c97271c15b92"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:59.229593 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.229568    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-openshift-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:59.231363 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.231336    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-xks-ca" pod "5567fb4d-f425-4536-ba0c-c97271c15b92" (UID: "5567fb4d-f425-4536-ba0c-c97271c15b92"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:59.231363 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.231354    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5567fb4d-f425-4536-ba0c-c97271c15b92-kube-api-access-vnpqm" pod "5567fb4d-f425-4536-ba0c-c97271c15b92" (UID: "5567fb4d-f425-4536-ba0c-c97271c15b92"). InnerVolumeSpecName "kube-api-access-vnpqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:59.330284 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.330249    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnpqm\" (UniqueName: \"kubernetes.io/projected/5567fb4d-f425-4536-ba0c-c97271c15b92-kube-api-access-vnpqm\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:59.330284 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.330280    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5567fb4d-f425-4536-ba0c-c97271c15b92-maas-api-xks-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:59.943327 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.943298    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"
Oct 30 22:13:59.943520 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.943347    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d" event={"ID":"5567fb4d-f425-4536-ba0c-c97271c15b92","Type":"ContainerDied","Data":"2932ffc516cef388d1822398d1247bbe61e714332b7dee5b7838833f4cc92ddd"}
Oct 30 22:13:59.943520 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:13:59.943395    5786 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2932ffc516cef388d1822398d1247bbe61e714332b7dee5b7838833f4cc92ddd"
Oct 30 22:14:01.048322 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:01.048289    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"]
Oct 30 22:14:01.098055 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:01.098014    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-4csqs"]
Oct 30 22:14:02.357888 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:02.357851    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a976f22c-8b5a-4592-b143-6ac83fd20559" path="/var/lib/kubelet/pods/a976f22c-8b5a-4592-b143-6ac83fd20559/volumes"
Oct 30 22:14:15.950447 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:15.950414    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"]
Oct 30 22:14:15.950906 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:15.950890    5786 cpu_manager.go:414] "containerMap: removing container" podUID="5567fb4d-f425-4536-ba0c-c97271c15b92" containerName="revoke-keys"
Oct 30 22:14:15.950906 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:15.950901    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5567fb4d-f425-4536-ba0c-c97271c15b92" containerName="revoke-keys"
Oct 30 22:14:15.951003 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:15.950993    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="5567fb4d-f425-4536-ba0c-c97271c15b92" containerName="revoke-keys"
Oct 30 22:14:15.954477 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:15.954456    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.003695 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.003654    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-socket\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.003991 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.003970    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt2f6\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-kube-api-access-xt2f6\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.004128 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.004114    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-credential-socket\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.004251 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.004235    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-token\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.004361 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.004347    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-podinfo\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.004496 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.004483    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-envoy\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.004633 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.004617    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eebfb660-3b45-4060-b8c1-c96772fb1187-istiod-ca-cert\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.004757 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.004744    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-data\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.004870 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.004855    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-certs\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.022263 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.022185    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-87bd1eb1-openshift-default-dockercfg-pxv4l\""
Oct 30 22:14:16.082268 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.082234    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"]
Oct 30 22:14:16.106327 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.106286    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-socket\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.106489 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.106384    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt2f6\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-kube-api-access-xt2f6\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.106489 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.106422    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-credential-socket\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.106615 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.106568    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-token\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.106748 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.106727    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-podinfo\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.107050 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.106827    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-socket\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.107519 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.107151    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-envoy\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.107904 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.107882    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eebfb660-3b45-4060-b8c1-c96772fb1187-istiod-ca-cert\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.108063 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.108046    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-data\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.108193 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.108177    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-certs\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.108554 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.108533    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-certs\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.108690 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.108666    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-data\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.108949 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.108923    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eebfb660-3b45-4060-b8c1-c96772fb1187-istiod-ca-cert\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.109040 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.108981    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-credential-socket\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.109820 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.109793    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-podinfo\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.110036 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.110014    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-envoy\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.162930 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.162893    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-token\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.163086 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.163032    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt2f6\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-kube-api-access-xt2f6\") pod \"e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") " pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.267471 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.267386    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:16.694485 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.694457    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"]
Oct 30 22:14:16.697247 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:14:16.697191    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebfb660_3b45_4060_b8c1_c96772fb1187.slice/crio-6634f65b04d898e2d019c973e6af3b6e8faf18a8c112e9ed02402f22cf625e6f WatchSource:0}: Error finding container 6634f65b04d898e2d019c973e6af3b6e8faf18a8c112e9ed02402f22cf625e6f: Status 404 returned error can't find the container with id 6634f65b04d898e2d019c973e6af3b6e8faf18a8c112e9ed02402f22cf625e6f
Oct 30 22:14:16.699506 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.699471    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:14:16.699593 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.699534    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:14:16.699593 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:16.699563    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:14:17.036504 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.036421    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" event={"ID":"eebfb660-3b45-4060-b8c1-c96772fb1187","Type":"ContainerStarted","Data":"b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618"}
Oct 30 22:14:17.036504 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.036477    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" event={"ID":"eebfb660-3b45-4060-b8c1-c96772fb1187","Type":"ContainerStarted","Data":"6634f65b04d898e2d019c973e6af3b6e8faf18a8c112e9ed02402f22cf625e6f"}
Oct 30 22:14:17.159876 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.159813    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" podStartSLOduration=2.159792783 podStartE2EDuration="2.159792783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:17.159439688 +0000 UTC m=+3713.030558824" watchObservedRunningTime="2026-10-30 22:14:17.159792783 +0000 UTC m=+3713.030911905"
Oct 30 22:14:17.268200 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.268159    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:17.272890 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.272866    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:17.314879 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.314787    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"]
Oct 30 22:14:17.318389 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.318367    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.381170 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.381131    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 22:14:17.381170 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.381169    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 22:14:17.381444 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.381232    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wqlw4\""
Oct 30 22:14:17.386804 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.386782    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:14:17.386953 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.386807    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:14:17.390149 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.390118    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"]
Oct 30 22:14:17.420352 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.420314    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d0c6301-a15c-4879-92a3-973c6a5f5d94-maas-api-tls\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.420519 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.420355    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d0c6301-a15c-4879-92a3-973c6a5f5d94-openshift-service-ca\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.420519 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.420383    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsgv8\" (UniqueName: \"kubernetes.io/projected/7d0c6301-a15c-4879-92a3-973c6a5f5d94-kube-api-access-fsgv8\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.420519 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.420416    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d0c6301-a15c-4879-92a3-973c6a5f5d94-metrics-certs\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.521178 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.521144    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d0c6301-a15c-4879-92a3-973c6a5f5d94-maas-api-tls\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.521178 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.521177    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d0c6301-a15c-4879-92a3-973c6a5f5d94-openshift-service-ca\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.521454 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.521198    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsgv8\" (UniqueName: \"kubernetes.io/projected/7d0c6301-a15c-4879-92a3-973c6a5f5d94-kube-api-access-fsgv8\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.521454 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.521249    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d0c6301-a15c-4879-92a3-973c6a5f5d94-metrics-certs\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.521979 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.521907    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d0c6301-a15c-4879-92a3-973c6a5f5d94-openshift-service-ca\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.523809 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.523782    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d0c6301-a15c-4879-92a3-973c6a5f5d94-metrics-certs\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.523938 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.523914    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d0c6301-a15c-4879-92a3-973c6a5f5d94-maas-api-tls\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.578895 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.578827    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsgv8\" (UniqueName: \"kubernetes.io/projected/7d0c6301-a15c-4879-92a3-973c6a5f5d94-kube-api-access-fsgv8\") pod \"maas-api-5fb4c4694-qrm8j\" (UID: \"7d0c6301-a15c-4879-92a3-973c6a5f5d94\") " pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:17.632931 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:17.632900    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:18.007494 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:18.007467    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"]
Oct 30 22:14:18.010085 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:14:18.010051    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0c6301_a15c_4879_92a3_973c6a5f5d94.slice/crio-81b5b8dd391a3562a3f4a1e4bae9eba99837171af8415767e6dff8bf32cd421f WatchSource:0}: Error finding container 81b5b8dd391a3562a3f4a1e4bae9eba99837171af8415767e6dff8bf32cd421f: Status 404 returned error can't find the container with id 81b5b8dd391a3562a3f4a1e4bae9eba99837171af8415767e6dff8bf32cd421f
Oct 30 22:14:18.011597 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:18.011576    5786 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:14:18.043364 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:18.043329    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j" event={"ID":"7d0c6301-a15c-4879-92a3-973c6a5f5d94","Type":"ContainerStarted","Data":"81b5b8dd391a3562a3f4a1e4bae9eba99837171af8415767e6dff8bf32cd421f"}
Oct 30 22:14:18.043721 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:18.043647    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:18.044593 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:18.044576    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:19.049821 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:19.049779    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j" event={"ID":"7d0c6301-a15c-4879-92a3-973c6a5f5d94","Type":"ContainerStarted","Data":"8a1ac7dcc3fb13302a3dd5b946ff6353000dafd31b39525c86cd7451db51125f"}
Oct 30 22:14:19.050311 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:19.050102    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:19.166414 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:19.166361    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j" podStartSLOduration=1.320352176 podStartE2EDuration="2.16634592s" totalImagesPullingTime="845.993744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:19.164278899 +0000 UTC m=+3715.035398040" watchObservedRunningTime="2026-10-30 22:14:19.16634592 +0000 UTC m=+3715.037465040"
Oct 30 22:14:22.354320 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:14:22.354285    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:14:23.159248 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.159190    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"]
Oct 30 22:14:23.164000 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.163970    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:23.220774 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.220742    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"]
Oct 30 22:14:23.253632 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.253601    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-87bd1eb1\""
Oct 30 22:14:23.253804 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.253648    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-87bd1eb1-dockercfg-nzxjb\""
Oct 30 22:14:23.284906 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.284870    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-plugins-config-volume\") pod \"payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2\" (UID: \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\") " pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:23.285079 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.284928    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxw5m\" (UniqueName: \"kubernetes.io/projected/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-kube-api-access-mxw5m\") pod \"payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2\" (UID: \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\") " pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:23.319134 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.319100    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"]
Oct 30 22:14:23.322678 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.322659    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:23.382298 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.382270    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"]
Oct 30 22:14:23.386354 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.386325    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-plugins-config-volume\") pod \"payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2\" (UID: \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\") " pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:23.386599 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.386574    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxw5m\" (UniqueName: \"kubernetes.io/projected/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-kube-api-access-mxw5m\") pod \"payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2\" (UID: \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\") " pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:23.387145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.387120    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-plugins-config-volume\") pod \"payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2\" (UID: \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\") " pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:23.444103 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.444068    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxw5m\" (UniqueName: \"kubernetes.io/projected/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-kube-api-access-mxw5m\") pod \"payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2\" (UID: \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\") " pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:23.478990 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.478958    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:23.488652 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.488287    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da08758f-8c5c-41ba-bb96-9d12308549bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl\" (UID: \"da08758f-8c5c-41ba-bb96-9d12308549bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:23.488652 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.488328    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2j4c\" (UniqueName: \"kubernetes.io/projected/da08758f-8c5c-41ba-bb96-9d12308549bc-kube-api-access-m2j4c\") pod \"payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl\" (UID: \"da08758f-8c5c-41ba-bb96-9d12308549bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:23.589834 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.589792    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da08758f-8c5c-41ba-bb96-9d12308549bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl\" (UID: \"da08758f-8c5c-41ba-bb96-9d12308549bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:23.589834 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.589839    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2j4c\" (UniqueName: \"kubernetes.io/projected/da08758f-8c5c-41ba-bb96-9d12308549bc-kube-api-access-m2j4c\") pod \"payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl\" (UID: \"da08758f-8c5c-41ba-bb96-9d12308549bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:23.590514 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.590495    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da08758f-8c5c-41ba-bb96-9d12308549bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl\" (UID: \"da08758f-8c5c-41ba-bb96-9d12308549bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:23.646698 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.646668    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2j4c\" (UniqueName: \"kubernetes.io/projected/da08758f-8c5c-41ba-bb96-9d12308549bc-kube-api-access-m2j4c\") pod \"payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl\" (UID: \"da08758f-8c5c-41ba-bb96-9d12308549bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:23.851981 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.851957    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"]
Oct 30 22:14:23.854694 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:14:23.854662    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51cdab10_cb20_44f3_a354_dd2aaaf17ea8.slice/crio-7b1e9ec1889524b7ab6b277d2ac3dc4678492cc0e0d9833412f88da9c07f6f09 WatchSource:0}: Error finding container 7b1e9ec1889524b7ab6b277d2ac3dc4678492cc0e0d9833412f88da9c07f6f09: Status 404 returned error can't find the container with id 7b1e9ec1889524b7ab6b277d2ac3dc4678492cc0e0d9833412f88da9c07f6f09
Oct 30 22:14:23.936194 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:23.936170    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:24.077252 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:24.077216    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2" event={"ID":"51cdab10-cb20-44f3-a354-dd2aaaf17ea8","Type":"ContainerStarted","Data":"45be5c6e6a391c9b1d42368317c8ef2ffadd03e462806ad591bed37b6453bd5d"}
Oct 30 22:14:24.077456 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:24.077275    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2" event={"ID":"51cdab10-cb20-44f3-a354-dd2aaaf17ea8","Type":"ContainerStarted","Data":"7b1e9ec1889524b7ab6b277d2ac3dc4678492cc0e0d9833412f88da9c07f6f09"}
Oct 30 22:14:24.077456 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:24.077362    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:24.193127 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:24.193068    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2" podStartSLOduration=1.1930532010000001 podStartE2EDuration="1.193053201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:24.192271346 +0000 UTC m=+3720.063390467" watchObservedRunningTime="2026-10-30 22:14:24.193053201 +0000 UTC m=+3720.064172321"
Oct 30 22:14:24.321509 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:14:24.321441    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda08758f_8c5c_41ba_bb96_9d12308549bc.slice/crio-7112d73e218367dacc28406d34060470228c2634b3e2803ac3eacf7a57a2e89a WatchSource:0}: Error finding container 7112d73e218367dacc28406d34060470228c2634b3e2803ac3eacf7a57a2e89a: Status 404 returned error can't find the container with id 7112d73e218367dacc28406d34060470228c2634b3e2803ac3eacf7a57a2e89a
Oct 30 22:14:24.324082 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:24.324045    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"]
Oct 30 22:14:25.060671 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:25.060639    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5fb4c4694-qrm8j"
Oct 30 22:14:25.083766 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:25.083733    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl" event={"ID":"da08758f-8c5c-41ba-bb96-9d12308549bc","Type":"ContainerStarted","Data":"2483c849be0c27b038376b50da566053606c92ac27a96f02ed7141b552ab41b6"}
Oct 30 22:14:25.083974 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:25.083791    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl" event={"ID":"da08758f-8c5c-41ba-bb96-9d12308549bc","Type":"ContainerStarted","Data":"7112d73e218367dacc28406d34060470228c2634b3e2803ac3eacf7a57a2e89a"}
Oct 30 22:14:25.083974 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:25.083963    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:25.300893 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:25.300838    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl" podStartSLOduration=2.300823478 podStartE2EDuration="2.300823478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:25.298827491 +0000 UTC m=+3721.169946612" watchObservedRunningTime="2026-10-30 22:14:25.300823478 +0000 UTC m=+3721.171942597"
Oct 30 22:14:35.087123 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:35.087095    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:14:36.092734 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:36.092703    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:14:42.182708 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:42.182673    5786 generic.go:356] "Generic (PLEG): container finished" podID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerID="b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618" exitCode=0
Oct 30 22:14:42.183182 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:42.182721    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" event={"ID":"eebfb660-3b45-4060-b8c1-c96772fb1187","Type":"ContainerDied","Data":"b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618"}
Oct 30 22:14:42.183182 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:42.183170    5786 scope.go:122] "RemoveContainer" containerID="b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618"
Oct 30 22:14:42.184483 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:42.184456    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:14:42.184547 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:42.184527    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:14:42.184590 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:42.184559    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:14:43.190387 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:43.190355    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" event={"ID":"eebfb660-3b45-4060-b8c1-c96772fb1187","Type":"ContainerStarted","Data":"be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2"}
Oct 30 22:14:43.268165 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:43.268129    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:43.268598 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:43.268504    5786 patch_prober.go:29] interesting pod/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 22:14:43.268843 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:43.268769    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 22:14:44.267986 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:44.267952    5786 patch_prober.go:29] interesting pod/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 22:14:44.268611 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:44.268582    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 22:14:45.270687 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:45.270645    5786 patch_prober.go:29] interesting pod/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 22:14:45.271747 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:45.271323    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 22:14:46.275233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:46.274628    5786 patch_prober.go:29] interesting pod/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 22:14:46.275233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:46.274691    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 22:14:47.275234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:47.274452    5786 patch_prober.go:29] interesting pod/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 22:14:47.275234 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:47.274528    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 22:14:48.043775 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:48.043735    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:48.271953 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:48.271921    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:48.272937 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:48.272914    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:14:52.354222 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:14:52.354172    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:14:57.057283 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:57.057246    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"]
Oct 30 22:14:57.165370 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:14:57.165335    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"]
Oct 30 22:15:00.169266 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.169229    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"]
Oct 30 22:15:00.173022 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.173008    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:00.225165 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.225138    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-crcdl\""
Oct 30 22:15:00.258158 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.258129    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jd6n\" (UniqueName: \"kubernetes.io/projected/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-kube-api-access-8jd6n\") pod \"maas-api-key-cleanup-29889975-rshw6\" (UID: \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:00.258323 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.258263    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-rshw6\" (UID: \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:00.276357 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.276328    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"]
Oct 30 22:15:00.359459 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.359423    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-rshw6\" (UID: \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:00.359668 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.359557    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jd6n\" (UniqueName: \"kubernetes.io/projected/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-kube-api-access-8jd6n\") pod \"maas-api-key-cleanup-29889975-rshw6\" (UID: \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:00.360049 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.360030    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-rshw6\" (UID: \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:00.416926 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.416888    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jd6n\" (UniqueName: \"kubernetes.io/projected/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-kube-api-access-8jd6n\") pod \"maas-api-key-cleanup-29889975-rshw6\" (UID: \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:00.486285 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.486179    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:00.862587 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:00.862506    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"]
Oct 30 22:15:00.864680 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:15:00.864654    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode655e8b9_8823_4dc8_93c8_cc3e0ffab3b9.slice/crio-0a5461c0d87ef6bbc08dc34a8ce355fee5d33221e5c462d589fd9ad884f8ee96 WatchSource:0}: Error finding container 0a5461c0d87ef6bbc08dc34a8ce355fee5d33221e5c462d589fd9ad884f8ee96: Status 404 returned error can't find the container with id 0a5461c0d87ef6bbc08dc34a8ce355fee5d33221e5c462d589fd9ad884f8ee96
Oct 30 22:15:01.293680 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:01.293642    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" event={"ID":"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9","Type":"ContainerStarted","Data":"b0daf6f93b3ca5f15c7d8cc823a00ae9283619d6f1870bff0072f7a1a6841e5f"}
Oct 30 22:15:01.294076 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:01.293705    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" event={"ID":"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9","Type":"ContainerStarted","Data":"0a5461c0d87ef6bbc08dc34a8ce355fee5d33221e5c462d589fd9ad884f8ee96"}
Oct 30 22:15:01.410025 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:01.409969    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" podStartSLOduration=1.409951823 podStartE2EDuration="1.409951823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:01.409299333 +0000 UTC m=+3757.280418455" watchObservedRunningTime="2026-10-30 22:15:01.409951823 +0000 UTC m=+3757.281070944"
Oct 30 22:15:02.057640 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.057594    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2" podUID="51cdab10-cb20-44f3-a354-dd2aaaf17ea8" containerName="payload-processing" containerID="cri-o://45be5c6e6a391c9b1d42368317c8ef2ffadd03e462806ad591bed37b6453bd5d" gracePeriod=25
Oct 30 22:15:02.165872 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.165815    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl" podUID="da08758f-8c5c-41ba-bb96-9d12308549bc" containerName="payload-pre-processing" containerID="cri-o://2483c849be0c27b038376b50da566053606c92ac27a96f02ed7141b552ab41b6" gracePeriod=25
Oct 30 22:15:02.301510 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.301478    5786 generic.go:356] "Generic (PLEG): container finished" podID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerID="b0daf6f93b3ca5f15c7d8cc823a00ae9283619d6f1870bff0072f7a1a6841e5f" exitCode=60
Oct 30 22:15:02.302012 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.301569    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" event={"ID":"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9","Type":"ContainerDied","Data":"b0daf6f93b3ca5f15c7d8cc823a00ae9283619d6f1870bff0072f7a1a6841e5f"}
Oct 30 22:15:02.302012 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.301826    5786 scope.go:122] "RemoveContainer" containerID="b0daf6f93b3ca5f15c7d8cc823a00ae9283619d6f1870bff0072f7a1a6841e5f"
Oct 30 22:15:02.303386 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.303352    5786 generic.go:356] "Generic (PLEG): container finished" podID="da08758f-8c5c-41ba-bb96-9d12308549bc" containerID="2483c849be0c27b038376b50da566053606c92ac27a96f02ed7141b552ab41b6" exitCode=0
Oct 30 22:15:02.303460 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.303422    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl" event={"ID":"da08758f-8c5c-41ba-bb96-9d12308549bc","Type":"ContainerDied","Data":"2483c849be0c27b038376b50da566053606c92ac27a96f02ed7141b552ab41b6"}
Oct 30 22:15:02.305134 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.305116    5786 generic.go:356] "Generic (PLEG): container finished" podID="51cdab10-cb20-44f3-a354-dd2aaaf17ea8" containerID="45be5c6e6a391c9b1d42368317c8ef2ffadd03e462806ad591bed37b6453bd5d" exitCode=0
Oct 30 22:15:02.305253 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.305156    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2" event={"ID":"51cdab10-cb20-44f3-a354-dd2aaaf17ea8","Type":"ContainerDied","Data":"45be5c6e6a391c9b1d42368317c8ef2ffadd03e462806ad591bed37b6453bd5d"}
Oct 30 22:15:02.347889 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.347864    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:15:02.379690 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.379663    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-kube-api-access-mxw5m\" (UniqueName: \"kubernetes.io/projected/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-kube-api-access-mxw5m\") pod \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\" (UID: \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\") "
Oct 30 22:15:02.379856 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.379713    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-plugins-config-volume\") pod \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\" (UID: \"51cdab10-cb20-44f3-a354-dd2aaaf17ea8\") "
Oct 30 22:15:02.380501 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.380138    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-plugins-config-volume" pod "51cdab10-cb20-44f3-a354-dd2aaaf17ea8" (UID: "51cdab10-cb20-44f3-a354-dd2aaaf17ea8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:02.380501 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.380302    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.452138 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.452098    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-kube-api-access-mxw5m" pod "51cdab10-cb20-44f3-a354-dd2aaaf17ea8" (UID: "51cdab10-cb20-44f3-a354-dd2aaaf17ea8"). InnerVolumeSpecName "kube-api-access-mxw5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:02.476313 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.476291    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:15:02.481595 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.481577    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxw5m\" (UniqueName: \"kubernetes.io/projected/51cdab10-cb20-44f3-a354-dd2aaaf17ea8-kube-api-access-mxw5m\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.582543 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.582518    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da08758f-8c5c-41ba-bb96-9d12308549bc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da08758f-8c5c-41ba-bb96-9d12308549bc-plugins-config-volume\") pod \"da08758f-8c5c-41ba-bb96-9d12308549bc\" (UID: \"da08758f-8c5c-41ba-bb96-9d12308549bc\") "
Oct 30 22:15:02.582663 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.582592    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da08758f-8c5c-41ba-bb96-9d12308549bc-kube-api-access-m2j4c\" (UniqueName: \"kubernetes.io/projected/da08758f-8c5c-41ba-bb96-9d12308549bc-kube-api-access-m2j4c\") pod \"da08758f-8c5c-41ba-bb96-9d12308549bc\" (UID: \"da08758f-8c5c-41ba-bb96-9d12308549bc\") "
Oct 30 22:15:02.582927 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.582905    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da08758f-8c5c-41ba-bb96-9d12308549bc-plugins-config-volume" pod "da08758f-8c5c-41ba-bb96-9d12308549bc" (UID: "da08758f-8c5c-41ba-bb96-9d12308549bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:02.584866 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.584842    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da08758f-8c5c-41ba-bb96-9d12308549bc-kube-api-access-m2j4c" pod "da08758f-8c5c-41ba-bb96-9d12308549bc" (UID: "da08758f-8c5c-41ba-bb96-9d12308549bc"). InnerVolumeSpecName "kube-api-access-m2j4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:02.683932 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.683903    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2j4c\" (UniqueName: \"kubernetes.io/projected/da08758f-8c5c-41ba-bb96-9d12308549bc-kube-api-access-m2j4c\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.683932 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:02.683930    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da08758f-8c5c-41ba-bb96-9d12308549bc-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:03.312043 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.311963    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"
Oct 30 22:15:03.312043 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.311980    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2" event={"ID":"51cdab10-cb20-44f3-a354-dd2aaaf17ea8","Type":"ContainerDied","Data":"7b1e9ec1889524b7ab6b277d2ac3dc4678492cc0e0d9833412f88da9c07f6f09"}
Oct 30 22:15:03.312584 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.312051    5786 scope.go:122] "RemoveContainer" containerID="45be5c6e6a391c9b1d42368317c8ef2ffadd03e462806ad591bed37b6453bd5d"
Oct 30 22:15:03.314235 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.314184    5786 generic.go:356] "Generic (PLEG): container finished" podID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerID="83ab941d94277acd7364894a8d0bb2b8d6e30cd4fd169212f0c5db36bbbea333" exitCode=60
Oct 30 22:15:03.314349 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.314257    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" event={"ID":"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9","Type":"ContainerDied","Data":"83ab941d94277acd7364894a8d0bb2b8d6e30cd4fd169212f0c5db36bbbea333"}
Oct 30 22:15:03.314542 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.314522    5786 scope.go:122] "RemoveContainer" containerID="83ab941d94277acd7364894a8d0bb2b8d6e30cd4fd169212f0c5db36bbbea333"
Oct 30 22:15:03.314777 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:15:03.314754    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-rshw6_odh-ai-gateway-infra(e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9"
Oct 30 22:15:03.315823 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.315804    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"
Oct 30 22:15:03.315954 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.315823    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl" event={"ID":"da08758f-8c5c-41ba-bb96-9d12308549bc","Type":"ContainerDied","Data":"7112d73e218367dacc28406d34060470228c2634b3e2803ac3eacf7a57a2e89a"}
Oct 30 22:15:03.324676 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.324377    5786 scope.go:122] "RemoveContainer" containerID="b0daf6f93b3ca5f15c7d8cc823a00ae9283619d6f1870bff0072f7a1a6841e5f"
Oct 30 22:15:03.338944 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.338925    5786 scope.go:122] "RemoveContainer" containerID="2483c849be0c27b038376b50da566053606c92ac27a96f02ed7141b552ab41b6"
Oct 30 22:15:03.474086 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.474053    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"]
Oct 30 22:15:03.477773 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.477749    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-87bd1eb1-d84d9876c-z4gh2"]
Oct 30 22:15:03.705553 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.705520    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"]
Oct 30 22:15:03.708080 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:03.708056    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-87bd1eb1-7f9b4f7d67-flcdl"]
Oct 30 22:15:04.324221 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:04.324173    5786 scope.go:122] "RemoveContainer" containerID="83ab941d94277acd7364894a8d0bb2b8d6e30cd4fd169212f0c5db36bbbea333"
Oct 30 22:15:04.324643 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:15:04.324477    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-rshw6_odh-ai-gateway-infra(e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9"
Oct 30 22:15:04.358719 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:04.358694    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51cdab10-cb20-44f3-a354-dd2aaaf17ea8" path="/var/lib/kubelet/pods/51cdab10-cb20-44f3-a354-dd2aaaf17ea8/volumes"
Oct 30 22:15:04.359081 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:04.359067    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da08758f-8c5c-41ba-bb96-9d12308549bc" path="/var/lib/kubelet/pods/da08758f-8c5c-41ba-bb96-9d12308549bc/volumes"
Oct 30 22:15:11.636771 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:11.636728    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"]
Oct 30 22:15:11.637287 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:11.637001    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy" containerID="cri-o://be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2" gracePeriod=30
Oct 30 22:15:13.389943 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.389908    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"]
Oct 30 22:15:13.390639 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.390616    5786 cpu_manager.go:414] "containerMap: removing container" podUID="51cdab10-cb20-44f3-a354-dd2aaaf17ea8" containerName="payload-processing"
Oct 30 22:15:13.390639 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.390638    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51cdab10-cb20-44f3-a354-dd2aaaf17ea8" containerName="payload-processing"
Oct 30 22:15:13.390836 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.390672    5786 cpu_manager.go:414] "containerMap: removing container" podUID="da08758f-8c5c-41ba-bb96-9d12308549bc" containerName="payload-pre-processing"
Oct 30 22:15:13.390836 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.390680    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da08758f-8c5c-41ba-bb96-9d12308549bc" containerName="payload-pre-processing"
Oct 30 22:15:13.390836 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.390806    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="51cdab10-cb20-44f3-a354-dd2aaaf17ea8" containerName="payload-processing"
Oct 30 22:15:13.390836 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.390819    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="da08758f-8c5c-41ba-bb96-9d12308549bc" containerName="payload-pre-processing"
Oct 30 22:15:13.398025 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.398000    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.465443 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.465416    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-6e7187-openshift-default-dockercfg-vzpqg\""
Oct 30 22:15:13.502121 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502088    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-podinfo\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.502313 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502225    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istiod-ca-cert\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.502313 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502252    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-token\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.502391 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502341    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-credential-socket\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.502476 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502457    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-data\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.502518 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502491    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-certs\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.502595 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502582    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-socket\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.502631 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502614    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-envoy\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.502668 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.502631    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zwbz\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-kube-api-access-2zwbz\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.520157 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.520122    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"]
Oct 30 22:15:13.604083 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604047    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-podinfo\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604321 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604125    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istiod-ca-cert\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604321 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604145    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-token\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604321 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604163    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-credential-socket\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604520 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604357    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-data\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604520 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604400    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-certs\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604520 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604474    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-socket\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604520 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604510    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-envoy\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604716 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604543    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zwbz\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-kube-api-access-2zwbz\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604716 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604654    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-credential-socket\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604832 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604754    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-data\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.604913 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604882    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-socket\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.605045 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.604951    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-certs\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.605045 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.605019    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istiod-ca-cert\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.606802 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.606783    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-podinfo\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.606981 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.606963    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-envoy\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.662328 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.662257    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zwbz\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-kube-api-access-2zwbz\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.662472 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.662342    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-token\") pod \"e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") " pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:13.721803 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:13.721768    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:14.148626 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:14.148598    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"]
Oct 30 22:15:14.150667 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:15:14.150637    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f7d12d8_d801_4b9d_9d1a_cac6c39f442f.slice/crio-54a8f959458ccd798d9e380f6b7352b25406d3a3509df14b788b538790954481 WatchSource:0}: Error finding container 54a8f959458ccd798d9e380f6b7352b25406d3a3509df14b788b538790954481: Status 404 returned error can't find the container with id 54a8f959458ccd798d9e380f6b7352b25406d3a3509df14b788b538790954481
Oct 30 22:15:14.152790 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:14.152759    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:15:14.152871 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:14.152825    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:15:14.152909 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:14.152876    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:15:14.398361 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:14.398258    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" event={"ID":"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f","Type":"ContainerStarted","Data":"fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2"}
Oct 30 22:15:14.398361 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:14.398353    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" event={"ID":"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f","Type":"ContainerStarted","Data":"54a8f959458ccd798d9e380f6b7352b25406d3a3509df14b788b538790954481"}
Oct 30 22:15:14.517620 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:14.517572    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" podStartSLOduration=1.517551267 podStartE2EDuration="1.517551267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:14.514907838 +0000 UTC m=+3770.386026959" watchObservedRunningTime="2026-10-30 22:15:14.517551267 +0000 UTC m=+3770.388670378"
Oct 30 22:15:14.722049 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:14.722018    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:15.727027 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:15.726994    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:16.408503 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:16.408471    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:16.409579 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:16.409551    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:16.709178 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:15:16.709138    5786 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebfb660_3b45_4060_b8c1_c96772fb1187.slice/crio-conmon-be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:15:16.899260 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:16.899231    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:15:17.040829 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.040757    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-credential-socket\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.040829 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.040821    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-data\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.041036 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.040878    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-socket\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.041036 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.040943    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-kube-api-access-xt2f6\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-kube-api-access-xt2f6\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.041036 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.040998    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-token\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-token\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.041197 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041046    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-certs\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.041197 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041076    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-envoy\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.041197 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041083    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-credential-socket" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:17.041197 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041120    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eebfb660-3b45-4060-b8c1-c96772fb1187-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eebfb660-3b45-4060-b8c1-c96772fb1187-istiod-ca-cert\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.041197 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041153    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-data" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:17.041197 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041176    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-podinfo\") pod \"eebfb660-3b45-4060-b8c1-c96772fb1187\" (UID: \"eebfb660-3b45-4060-b8c1-c96772fb1187\") "
Oct 30 22:15:17.041581 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041197    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-socket" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:17.041581 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041496    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eebfb660-3b45-4060-b8c1-c96772fb1187-istiod-ca-cert" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:17.041581 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041349    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-certs" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:17.041752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041698    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.041752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041716    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.041752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041730    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.041752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041743    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.041752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.041755    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eebfb660-3b45-4060-b8c1-c96772fb1187-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.043537 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.043511    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-envoy" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:17.043709 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.043688    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-kube-api-access-xt2f6" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "kube-api-access-xt2f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:17.043966 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.043941    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-token" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:17.043966 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.043943    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-podinfo" pod "eebfb660-3b45-4060-b8c1-c96772fb1187" (UID: "eebfb660-3b45-4060-b8c1-c96772fb1187"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:17.143233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.143175    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.143233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.143226    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt2f6\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-kube-api-access-xt2f6\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.143233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.143238    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.143483 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.143246    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eebfb660-3b45-4060-b8c1-c96772fb1187-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.352893 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.352815    5786 scope.go:122] "RemoveContainer" containerID="83ab941d94277acd7364894a8d0bb2b8d6e30cd4fd169212f0c5db36bbbea333"
Oct 30 22:15:17.416178 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.416146    5786 generic.go:356] "Generic (PLEG): container finished" podID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerID="be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2" exitCode=0
Oct 30 22:15:17.416344 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.416223    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"
Oct 30 22:15:17.416344 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.416236    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" event={"ID":"eebfb660-3b45-4060-b8c1-c96772fb1187","Type":"ContainerDied","Data":"be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2"}
Oct 30 22:15:17.416344 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.416299    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s" event={"ID":"eebfb660-3b45-4060-b8c1-c96772fb1187","Type":"ContainerDied","Data":"6634f65b04d898e2d019c973e6af3b6e8faf18a8c112e9ed02402f22cf625e6f"}
Oct 30 22:15:17.416344 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.416324    5786 scope.go:122] "RemoveContainer" containerID="be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2"
Oct 30 22:15:17.429015 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.428988    5786 scope.go:122] "RemoveContainer" containerID="b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618"
Oct 30 22:15:17.438990 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.438935    5786 scope.go:122] "RemoveContainer" containerID="be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2"
Oct 30 22:15:17.439329 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:15:17.439308    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2\": container with ID starting with be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2 not found: ID does not exist" containerID="be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2"
Oct 30 22:15:17.439406 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.439341    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2"} err="failed to get container status \"be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2\": rpc error: code = NotFound desc = could not find container \"be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2\": container with ID starting with be616c69ce077dfd96a0dc179c3234a07898eb26ac06d7a3ecab113525b2cfa2 not found: ID does not exist"
Oct 30 22:15:17.439406 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.439360    5786 scope.go:122] "RemoveContainer" containerID="b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618"
Oct 30 22:15:17.439628 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:15:17.439608    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618\": container with ID starting with b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618 not found: ID does not exist" containerID="b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618"
Oct 30 22:15:17.439688 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.439640    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618"} err="failed to get container status \"b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618\": rpc error: code = NotFound desc = could not find container \"b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618\": container with ID starting with b889b941b54d128b9a969c5025b72f3ef1cb6033d6cd49b035eb2d0b0a2c8618 not found: ID does not exist"
Oct 30 22:15:17.583818 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.583785    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"]
Oct 30 22:15:17.632837 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:17.632764    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-87bd1eb1-openshift-default-5788f96d77-rjb7s"]
Oct 30 22:15:18.358562 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:18.358474    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" path="/var/lib/kubelet/pods/eebfb660-3b45-4060-b8c1-c96772fb1187/volumes"
Oct 30 22:15:18.423674 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:18.423635    5786 generic.go:356] "Generic (PLEG): container finished" podID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerID="fb54ac54cbcd8095123c08159c787c5459d5ddeea1c8be990119b65f05de7f4e" exitCode=60
Oct 30 22:15:18.423888 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:18.423719    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" event={"ID":"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9","Type":"ContainerDied","Data":"fb54ac54cbcd8095123c08159c787c5459d5ddeea1c8be990119b65f05de7f4e"}
Oct 30 22:15:18.423888 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:18.423790    5786 scope.go:122] "RemoveContainer" containerID="83ab941d94277acd7364894a8d0bb2b8d6e30cd4fd169212f0c5db36bbbea333"
Oct 30 22:15:18.424218 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:18.424186    5786 scope.go:122] "RemoveContainer" containerID="fb54ac54cbcd8095123c08159c787c5459d5ddeea1c8be990119b65f05de7f4e"
Oct 30 22:15:18.424512 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:15:18.424492    5786 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-rshw6_odh-ai-gateway-infra(e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9"
Oct 30 22:15:19.561742 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:19.561704    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"]
Oct 30 22:15:19.697418 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:19.697395    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:19.771048 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:19.771025    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-kube-api-access-8jd6n\" (UniqueName: \"kubernetes.io/projected/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-kube-api-access-8jd6n\") pod \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\" (UID: \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\") "
Oct 30 22:15:19.771167 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:19.771154    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-maas-api-service-ca\") pod \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\" (UID: \"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9\") "
Oct 30 22:15:19.771484 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:19.771464    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-maas-api-service-ca" pod "e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" (UID: "e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:19.773323 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:19.773301    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-kube-api-access-8jd6n" pod "e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" (UID: "e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9"). InnerVolumeSpecName "kube-api-access-8jd6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:19.872715 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:19.872636    5786 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-maas-api-service-ca\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.872715 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:19.872661    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jd6n\" (UniqueName: \"kubernetes.io/projected/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9-kube-api-access-8jd6n\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:20.438609 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.438575    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"
Oct 30 22:15:20.438609 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.438592    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6" event={"ID":"e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9","Type":"ContainerDied","Data":"0a5461c0d87ef6bbc08dc34a8ce355fee5d33221e5c462d589fd9ad884f8ee96"}
Oct 30 22:15:20.438894 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.438664    5786 scope.go:122] "RemoveContainer" containerID="fb54ac54cbcd8095123c08159c787c5459d5ddeea1c8be990119b65f05de7f4e"
Oct 30 22:15:20.602921 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.602883    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"]
Oct 30 22:15:20.605032 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.605009    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-rshw6"]
Oct 30 22:15:20.960746 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.960705    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"]
Oct 30 22:15:20.961187 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961173    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:20.961267 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961189    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:20.961267 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961231    5786 cpu_manager.go:414] "containerMap: removing container" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy"
Oct 30 22:15:20.961267 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961236    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy"
Oct 30 22:15:20.961267 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961247    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:20.961267 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961252    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:20.961427 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961336    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:20.961427 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961355    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy"
Oct 30 22:15:20.961427 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961365    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy"
Oct 30 22:15:20.961427 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.961375    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:20.966025 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:20.966010    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:21.034233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.021657    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-6e7187-dockercfg-pt5kd\""
Oct 30 22:15:21.034233 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.024883    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"]
Oct 30 22:15:21.072539 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.072484    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-6e7187\""
Oct 30 22:15:21.088032 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.087996    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7\" (UID: \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:21.088231 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.088195    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc87n\" (UniqueName: \"kubernetes.io/projected/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-kube-api-access-fc87n\") pod \"payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7\" (UID: \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:21.114102 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.114076    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"]
Oct 30 22:15:21.114549 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.114536    5786 cpu_manager.go:414] "containerMap: removing container" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy"
Oct 30 22:15:21.114618 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.114551    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eebfb660-3b45-4060-b8c1-c96772fb1187" containerName="istio-proxy"
Oct 30 22:15:21.114618 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.114567    5786 cpu_manager.go:414] "containerMap: removing container" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:21.114618 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.114572    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:21.114770 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.114676    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" containerName="cleanup"
Oct 30 22:15:21.117871 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.117855    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:21.175704 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.175670    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"]
Oct 30 22:15:21.189538 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.189512    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc87n\" (UniqueName: \"kubernetes.io/projected/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-kube-api-access-fc87n\") pod \"payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7\" (UID: \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:21.189738 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.189564    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7\" (UID: \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:21.190278 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.190256    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7\" (UID: \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:21.245996 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.245936    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc87n\" (UniqueName: \"kubernetes.io/projected/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-kube-api-access-fc87n\") pod \"payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7\" (UID: \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:21.278915 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.278886    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:21.290441 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.290410    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsjw7\" (UniqueName: \"kubernetes.io/projected/953d11ed-8f69-4ce9-a81e-370ac067864a-kube-api-access-bsjw7\") pod \"payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb\" (UID: \"953d11ed-8f69-4ce9-a81e-370ac067864a\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:21.290587 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.290566    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953d11ed-8f69-4ce9-a81e-370ac067864a-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb\" (UID: \"953d11ed-8f69-4ce9-a81e-370ac067864a\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:21.391155 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.391123    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953d11ed-8f69-4ce9-a81e-370ac067864a-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb\" (UID: \"953d11ed-8f69-4ce9-a81e-370ac067864a\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:21.391386 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.391282    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsjw7\" (UniqueName: \"kubernetes.io/projected/953d11ed-8f69-4ce9-a81e-370ac067864a-kube-api-access-bsjw7\") pod \"payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb\" (UID: \"953d11ed-8f69-4ce9-a81e-370ac067864a\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:21.391956 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.391936    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953d11ed-8f69-4ce9-a81e-370ac067864a-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb\" (UID: \"953d11ed-8f69-4ce9-a81e-370ac067864a\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:21.448428 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.448396    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsjw7\" (UniqueName: \"kubernetes.io/projected/953d11ed-8f69-4ce9-a81e-370ac067864a-kube-api-access-bsjw7\") pod \"payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb\" (UID: \"953d11ed-8f69-4ce9-a81e-370ac067864a\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:21.651593 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.651569    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"]
Oct 30 22:15:21.654120 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:15:21.654090    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e587ffd_eb44_4774_a5d1_4d92d5cf71f8.slice/crio-97e757026e7f4d2b57bef6a552e01fae0ec6046c941b25939d0116fa139de8db WatchSource:0}: Error finding container 97e757026e7f4d2b57bef6a552e01fae0ec6046c941b25939d0116fa139de8db: Status 404 returned error can't find the container with id 97e757026e7f4d2b57bef6a552e01fae0ec6046c941b25939d0116fa139de8db
Oct 30 22:15:21.731907 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:21.731881    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:22.110533 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.110503    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"]
Oct 30 22:15:22.114599 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:15:22.114568    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953d11ed_8f69_4ce9_a81e_370ac067864a.slice/crio-004fb20dfb81e1c42fca4eb6bf3f5580c9b5c42a1b587997e19c6f2732965895 WatchSource:0}: Error finding container 004fb20dfb81e1c42fca4eb6bf3f5580c9b5c42a1b587997e19c6f2732965895: Status 404 returned error can't find the container with id 004fb20dfb81e1c42fca4eb6bf3f5580c9b5c42a1b587997e19c6f2732965895
Oct 30 22:15:22.354612 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:15:22.354574    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:15:22.358651 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.358623    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9" path="/var/lib/kubelet/pods/e655e8b9-8823-4dc8-93c8-cc3e0ffab3b9/volumes"
Oct 30 22:15:22.452135 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.452104    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb" event={"ID":"953d11ed-8f69-4ce9-a81e-370ac067864a","Type":"ContainerStarted","Data":"c15595b41541a4567cda628afef0144543b98c4f724b8e00d8ef72323c86c60c"}
Oct 30 22:15:22.452368 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.452166    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:22.452368 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.452190    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb" event={"ID":"953d11ed-8f69-4ce9-a81e-370ac067864a","Type":"ContainerStarted","Data":"004fb20dfb81e1c42fca4eb6bf3f5580c9b5c42a1b587997e19c6f2732965895"}
Oct 30 22:15:22.453769 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.453741    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7" event={"ID":"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8","Type":"ContainerStarted","Data":"c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46"}
Oct 30 22:15:22.453904 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.453793    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7" event={"ID":"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8","Type":"ContainerStarted","Data":"97e757026e7f4d2b57bef6a552e01fae0ec6046c941b25939d0116fa139de8db"}
Oct 30 22:15:22.453904 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.453822    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:22.569739 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:22.569679    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb" podStartSLOduration=1.5696607089999999 podStartE2EDuration="1.569660709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:22.568785497 +0000 UTC m=+3778.439904614" watchObservedRunningTime="2026-10-30 22:15:22.569660709 +0000 UTC m=+3778.440779836"
Oct 30 22:15:28.194857 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:28.194822    5786 scope.go:122] "RemoveContainer" containerID="505d3e7a9b76525d47fae877a6db6d9fb0a63bb87ca66f9d4e900ad0c6d56e27"
Oct 30 22:15:28.207983 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:28.207955    5786 scope.go:122] "RemoveContainer" containerID="709100e57714cd5d729cc4cc4e15fc257cf35fc431599b14fec69ef9c2209e3e"
Oct 30 22:15:33.477348 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:33.477265    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:15:33.477874 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:33.477350    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:15:33.592591 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:33.592538    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7" podStartSLOduration=13.592522805 podStartE2EDuration="13.592522805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:22.695286995 +0000 UTC m=+3778.566406116" watchObservedRunningTime="2026-10-30 22:15:33.592522805 +0000 UTC m=+3789.463641915"
Oct 30 22:15:41.554091 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:41.554056    5786 generic.go:356] "Generic (PLEG): container finished" podID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerID="fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2" exitCode=0
Oct 30 22:15:41.554628 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:41.554130    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" event={"ID":"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f","Type":"ContainerDied","Data":"fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2"}
Oct 30 22:15:41.554628 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:41.554531    5786 scope.go:122] "RemoveContainer" containerID="fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2"
Oct 30 22:15:41.555727 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:41.555701    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:15:41.555786 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:41.555768    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:15:41.555822 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:41.555801    5786 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:15:42.580798 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:42.580762    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" event={"ID":"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f","Type":"ContainerStarted","Data":"e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886"}
Oct 30 22:15:42.722109 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:42.722070    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:42.723291 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:42.723263    5786 patch_prober.go:29] interesting pod/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused" start-of-body=
Oct 30 22:15:42.723381 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:42.723328    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused"
Oct 30 22:15:43.722302 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:43.722268    5786 patch_prober.go:29] interesting pod/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused" start-of-body=
Oct 30 22:15:43.722730 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:43.722342    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused"
Oct 30 22:15:44.722471 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:44.722437    5786 patch_prober.go:29] interesting pod/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused" start-of-body=
Oct 30 22:15:44.722897 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:44.722502    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused"
Oct 30 22:15:45.722823 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:45.722789    5786 patch_prober.go:29] interesting pod/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused" start-of-body=
Oct 30 22:15:45.723274 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:45.722841    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused"
Oct 30 22:15:46.409255 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:46.409224    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:46.722499 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:46.722465    5786 patch_prober.go:29] interesting pod/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused" start-of-body=
Oct 30 22:15:46.722697 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:46.722517    5786 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.2:15021/healthz/ready\": dial tcp 10.134.1.2:15021: connect: connection refused"
Oct 30 22:15:47.756308 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:47.756184    5786 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:47.757744 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:47.757720    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:15:52.355145 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:15:52.355113    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:15:58.064536 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:58.064501    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"]
Oct 30 22:15:58.112909 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:58.112876    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bef2f275-b8hpm"]
Oct 30 22:15:58.358680 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:15:58.358600    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63232033-08b4-4b7d-a00b-3988df1d9077" path="/var/lib/kubelet/pods/63232033-08b4-4b7d-a00b-3988df1d9077/volumes"
Oct 30 22:16:22.354682 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:16:22.354642    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:16:52.355006 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:16:52.354968    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:17:16.996306 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:16.996228    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"]
Oct 30 22:17:17.104976 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:17.104944    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"]
Oct 30 22:17:21.997291 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:21.997247    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb" podUID="953d11ed-8f69-4ce9-a81e-370ac067864a" containerName="payload-processing" containerID="cri-o://c15595b41541a4567cda628afef0144543b98c4f724b8e00d8ef72323c86c60c" gracePeriod=25
Oct 30 22:17:22.105484 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.105447    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7" podUID="9e587ffd-eb44-4774-a5d1-4d92d5cf71f8" containerName="payload-pre-processing" containerID="cri-o://c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46" gracePeriod=25
Oct 30 22:17:22.116670 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.116488    5786 generic.go:356] "Generic (PLEG): container finished" podID="953d11ed-8f69-4ce9-a81e-370ac067864a" containerID="c15595b41541a4567cda628afef0144543b98c4f724b8e00d8ef72323c86c60c" exitCode=0
Oct 30 22:17:22.116670 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.116567    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb" event={"ID":"953d11ed-8f69-4ce9-a81e-370ac067864a","Type":"ContainerDied","Data":"c15595b41541a4567cda628afef0144543b98c4f724b8e00d8ef72323c86c60c"}
Oct 30 22:17:22.264600 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.264534    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:17:22.333941 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.333915    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/953d11ed-8f69-4ce9-a81e-370ac067864a-kube-api-access-bsjw7\" (UniqueName: \"kubernetes.io/projected/953d11ed-8f69-4ce9-a81e-370ac067864a-kube-api-access-bsjw7\") pod \"953d11ed-8f69-4ce9-a81e-370ac067864a\" (UID: \"953d11ed-8f69-4ce9-a81e-370ac067864a\") "
Oct 30 22:17:22.334155 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.334098    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/953d11ed-8f69-4ce9-a81e-370ac067864a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953d11ed-8f69-4ce9-a81e-370ac067864a-plugins-config-volume\") pod \"953d11ed-8f69-4ce9-a81e-370ac067864a\" (UID: \"953d11ed-8f69-4ce9-a81e-370ac067864a\") "
Oct 30 22:17:22.334612 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.334535    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953d11ed-8f69-4ce9-a81e-370ac067864a-plugins-config-volume" pod "953d11ed-8f69-4ce9-a81e-370ac067864a" (UID: "953d11ed-8f69-4ce9-a81e-370ac067864a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:22.336104 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.336078    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953d11ed-8f69-4ce9-a81e-370ac067864a-kube-api-access-bsjw7" pod "953d11ed-8f69-4ce9-a81e-370ac067864a" (UID: "953d11ed-8f69-4ce9-a81e-370ac067864a"). InnerVolumeSpecName "kube-api-access-bsjw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:22.353060 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.353037    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:17:22.357079 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:17:22.357056    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:17:22.435039 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.435013    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-plugins-config-volume\") pod \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\" (UID: \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\") "
Oct 30 22:17:22.435242 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.435080    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-kube-api-access-fc87n\" (UniqueName: \"kubernetes.io/projected/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-kube-api-access-fc87n\") pod \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\" (UID: \"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8\") "
Oct 30 22:17:22.435496 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.435481    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/953d11ed-8f69-4ce9-a81e-370ac067864a-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:22.435584 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.435498    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsjw7\" (UniqueName: \"kubernetes.io/projected/953d11ed-8f69-4ce9-a81e-370ac067864a-kube-api-access-bsjw7\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:22.435584 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.435479    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-plugins-config-volume" pod "9e587ffd-eb44-4774-a5d1-4d92d5cf71f8" (UID: "9e587ffd-eb44-4774-a5d1-4d92d5cf71f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:22.437434 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.437412    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-kube-api-access-fc87n" pod "9e587ffd-eb44-4774-a5d1-4d92d5cf71f8" (UID: "9e587ffd-eb44-4774-a5d1-4d92d5cf71f8"). InnerVolumeSpecName "kube-api-access-fc87n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:22.536254 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.536163    5786 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-plugins-config-volume\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:22.536254 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:22.536191    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc87n\" (UniqueName: \"kubernetes.io/projected/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8-kube-api-access-fc87n\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:23.122776 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.122747    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"
Oct 30 22:17:23.123298 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.122793    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb" event={"ID":"953d11ed-8f69-4ce9-a81e-370ac067864a","Type":"ContainerDied","Data":"004fb20dfb81e1c42fca4eb6bf3f5580c9b5c42a1b587997e19c6f2732965895"}
Oct 30 22:17:23.123298 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.122856    5786 scope.go:122] "RemoveContainer" containerID="c15595b41541a4567cda628afef0144543b98c4f724b8e00d8ef72323c86c60c"
Oct 30 22:17:23.124313 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.124290    5786 generic.go:356] "Generic (PLEG): container finished" podID="9e587ffd-eb44-4774-a5d1-4d92d5cf71f8" containerID="c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46" exitCode=0
Oct 30 22:17:23.124482 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.124334    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"
Oct 30 22:17:23.124482 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.124388    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7" event={"ID":"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8","Type":"ContainerDied","Data":"c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46"}
Oct 30 22:17:23.124482 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.124432    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7" event={"ID":"9e587ffd-eb44-4774-a5d1-4d92d5cf71f8","Type":"ContainerDied","Data":"97e757026e7f4d2b57bef6a552e01fae0ec6046c941b25939d0116fa139de8db"}
Oct 30 22:17:23.132466 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.132395    5786 scope.go:122] "RemoveContainer" containerID="c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46"
Oct 30 22:17:23.140379 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.140359    5786 scope.go:122] "RemoveContainer" containerID="c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46"
Oct 30 22:17:23.140608 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:17:23.140589    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46\": container with ID starting with c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46 not found: ID does not exist" containerID="c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46"
Oct 30 22:17:23.140675 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.140617    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46"} err="failed to get container status \"c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46\": rpc error: code = NotFound desc = could not find container \"c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46\": container with ID starting with c0fe9a92ca2593a1ed96c4635d47a54d3cf63e37875b21d6c81d8573b41d6c46 not found: ID does not exist"
Oct 30 22:17:23.284061 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.283979    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"]
Oct 30 22:17:23.333138 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.333106    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-6e7187-65f875994c-mrzh7"]
Oct 30 22:17:23.440851 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.440817    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"]
Oct 30 22:17:23.442637 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:23.442615    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-6e7187-948f99455-5kvzb"]
Oct 30 22:17:24.361515 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:24.359042    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="953d11ed-8f69-4ce9-a81e-370ac067864a" path="/var/lib/kubelet/pods/953d11ed-8f69-4ce9-a81e-370ac067864a/volumes"
Oct 30 22:17:24.361515 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:24.359647    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e587ffd-eb44-4774-a5d1-4d92d5cf71f8" path="/var/lib/kubelet/pods/9e587ffd-eb44-4774-a5d1-4d92d5cf71f8/volumes"
Oct 30 22:17:28.256290 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:28.256258    5786 scope.go:122] "RemoveContainer" containerID="aed01c86d34b90ebf95e71864e574ffa50d1f6254f34d65a44f495e1ce8f5f3b"
Oct 30 22:17:28.265139 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:28.265119    5786 scope.go:122] "RemoveContainer" containerID="3722d94db258cf2920fda1b1984df33202a67de358a5a56c4b3c1a37e8734429"
Oct 30 22:17:35.754141 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:35.754112    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"]
Oct 30 22:17:35.754686 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:35.754332    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy" containerID="cri-o://e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886" gracePeriod=30
Oct 30 22:17:41.021353 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.021280    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:17:41.047751 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.047719    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-podinfo\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.047924 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.047761    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istiod-ca-cert\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.047924 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.047840    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-token\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-token\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.047924 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.047859    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-credential-socket\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.047924 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.047900    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-certs\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.048173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.047978    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-socket\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.048173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048011    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-kube-api-access-2zwbz\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-kube-api-access-2zwbz\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.048173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048048    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-data\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.048173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048118    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-envoy\") pod \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\" (UID: \"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f\") "
Oct 30 22:17:41.048414 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048308    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-credential-socket" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:41.048414 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048326    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-certs" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:41.048414 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048355    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istiod-ca-cert" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:41.048548 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048530    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-socket" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:41.048548 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048541    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-data" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:41.048650 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048628    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.048695 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048659    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.048695 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.048675    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.050475 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.050445    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-envoy" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:41.050598 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.050574    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-kube-api-access-2zwbz" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "kube-api-access-2zwbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:41.050824 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.050800    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-podinfo" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:41.050824 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.050817    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-token" pod "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" (UID: "1f7d12d8-d801-4b9d-9d1a-cac6c39f442f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:41.149690 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.149654    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.149690 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.149687    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zwbz\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-kube-api-access-2zwbz\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.149690 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.149697    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.149992 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.149707    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.149992 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.149717    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.149992 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.149725    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:41.250574 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.250540    5786 generic.go:356] "Generic (PLEG): container finished" podID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerID="e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886" exitCode=0
Oct 30 22:17:41.250753 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.250593    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"
Oct 30 22:17:41.250753 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.250612    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" event={"ID":"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f","Type":"ContainerDied","Data":"e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886"}
Oct 30 22:17:41.250753 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.250665    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh" event={"ID":"1f7d12d8-d801-4b9d-9d1a-cac6c39f442f","Type":"ContainerDied","Data":"54a8f959458ccd798d9e380f6b7352b25406d3a3509df14b788b538790954481"}
Oct 30 22:17:41.250753 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.250687    5786 scope.go:122] "RemoveContainer" containerID="e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886"
Oct 30 22:17:41.260407 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.260356    5786 scope.go:122] "RemoveContainer" containerID="fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2"
Oct 30 22:17:41.268645 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.268624    5786 scope.go:122] "RemoveContainer" containerID="e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886"
Oct 30 22:17:41.268925 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:17:41.268907    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886\": container with ID starting with e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886 not found: ID does not exist" containerID="e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886"
Oct 30 22:17:41.268997 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.268939    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886"} err="failed to get container status \"e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886\": rpc error: code = NotFound desc = could not find container \"e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886\": container with ID starting with e676891835566fde83883b210c1d0ec4ae7143d69f3ee10d0acf2a8cfc3f8886 not found: ID does not exist"
Oct 30 22:17:41.268997 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.268966    5786 scope.go:122] "RemoveContainer" containerID="fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2"
Oct 30 22:17:41.269266 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:17:41.269249    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2\": container with ID starting with fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2 not found: ID does not exist" containerID="fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2"
Oct 30 22:17:41.269314 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.269273    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2"} err="failed to get container status \"fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2\": rpc error: code = NotFound desc = could not find container \"fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2\": container with ID starting with fa1c04f56f6dd02026c79d0ad7a8b56f4eca3f5707be1fd4efc84433bd9818f2 not found: ID does not exist"
Oct 30 22:17:41.418787 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.418755    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"]
Oct 30 22:17:41.425012 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:41.424979    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6e7187-openshift-default-5ccdd68489-kmzdh"]
Oct 30 22:17:42.358397 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:42.358364    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" path="/var/lib/kubelet/pods/1f7d12d8-d801-4b9d-9d1a-cac6c39f442f/volumes"
Oct 30 22:17:52.489586 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:17:52.489526    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:17:56.049456 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:56.049419    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"]
Oct 30 22:17:56.098709 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:56.098651    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-dmnh6"]
Oct 30 22:17:56.358265 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:17:56.358171    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="120df019-dde6-4b6b-b1c6-a44aa86746c8" path="/var/lib/kubelet/pods/120df019-dde6-4b6b-b1c6-a44aa86746c8/volumes"
Oct 30 22:18:22.355693 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:18:22.355655    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:18:29.736685 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:18:29.736405    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-6vwl6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.220:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:18:29.737355 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:18:29.737319    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.220:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:29.755231 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:18:29.754334    5786 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8b756 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.221:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:18:29.755231 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:18:29.754418    5786 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.221:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:31.047797 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:18:31.047755    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"]
Oct 30 22:18:31.096730 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:18:31.096698    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-fcslq"]
Oct 30 22:18:32.358353 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:18:32.358322    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23a9006c-c90d-482c-ae13-33d24b45d897" path="/var/lib/kubelet/pods/23a9006c-c90d-482c-ae13-33d24b45d897/volumes"
Oct 30 22:18:52.355669 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:18:52.355632    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:19:00.048310 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:00.048267    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"]
Oct 30 22:19:00.097255 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:00.097216    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec5c9b19-wzv4d"]
Oct 30 22:19:00.358196 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:00.358113    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5567fb4d-f425-4536-ba0c-c97271c15b92" path="/var/lib/kubelet/pods/5567fb4d-f425-4536-ba0c-c97271c15b92/volumes"
Oct 30 22:19:22.356074 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:19:22.356025    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:19:28.314650 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:28.314609    5786 scope.go:122] "RemoveContainer" containerID="936f5f1915df5d16273b58f4189935ebabc5653b4daed9baa595f66f76bec6ea"
Oct 30 22:19:28.323986 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:28.323963    5786 scope.go:122] "RemoveContainer" containerID="7b98bf8bd0d390793c7b3e9fdfc3e9d8d05c801b9e225b95f076f068c2beb6a6"
Oct 30 22:19:35.675220 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:35.675177    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-76d5869bc7-svshz_ec5111f7-df66-489c-8fe3-95c2a03aa914/manager/0.log"
Oct 30 22:19:37.809077 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:37.809041    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-7dx2p_98ff4001-c3d7-40d3-88a8-d0d801917b7e/manager/1.log"
Oct 30 22:19:38.630349 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:38.630310    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-7dx2p_98ff4001-c3d7-40d3-88a8-d0d801917b7e/manager/0.log"
Oct 30 22:19:39.219222 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:39.219168    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5fb4c4694-qrm8j_7d0c6301-a15c-4879-92a3-973c6a5f5d94/maas-api/0.log"
Oct 30 22:19:52.362121 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:19:52.362075    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:19:54.902644 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:19:54.902614    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-lbxmv_6b85ec40-a08a-4220-b02a-9959b72251e1/discovery/0.log"
Oct 30 22:20:05.282462 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:05.282371    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c8944d49-9c745_05b9e5e9-fd34-46c1-9688-062a0c091b51/kube-auth-proxy/0.log"
Oct 30 22:20:06.044739 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:06.044701    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-6vwl6_065139b3-6b4a-450b-a2ec-d8a8cf01caf3/istio-proxy/0.log"
Oct 30 22:20:06.319726 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:06.319645    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-8b756_15ae86a1-5c75-45ff-b549-32aa463d30a7/istio-proxy/0.log"
Oct 30 22:20:07.953744 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:07.953705    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-xtfjt_9ad04b60-4d70-44de-badb-f602b63d1556/istio-proxy/0.log"
Oct 30 22:20:08.238997 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:08.238917    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-zr2km_773d8fc2-85de-47dd-81b5-6831f66f5594/istio-proxy/0.log"
Oct 30 22:20:08.790478 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:08.790448    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-6hstp_65d3c06a-089a-4595-a2fe-0c4a769b49aa/payload-pre-processing/0.log"
Oct 30 22:20:09.035752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:09.035724    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:20:10.483561 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:10.483528    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-8687n_086aa9d4-d49b-4f59-a95f-2a6eb8a82928/main/0.log"
Oct 30 22:20:11.311564 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:11.311526    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb_b5613d2c-edac-4610-9c85-03a98b6a8493/main/0.log"
Oct 30 22:20:12.140056 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:12.140023    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs_6523484b-480f-4c20-b123-8013f0ef711e/main/0.log"
Oct 30 22:20:22.355116 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:20:22.355081    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:20:28.350525 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:28.350492    5786 scope.go:122] "RemoveContainer" containerID="542aefb0e826db80a4b8548122da8a98b7af2b8b55cf5686f5265afe614b317c"
Oct 30 22:20:32.272563 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:32.272532    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-76d5869bc7-svshz_ec5111f7-df66-489c-8fe3-95c2a03aa914/manager/0.log"
Oct 30 22:20:34.329755 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:34.329723    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-7dx2p_98ff4001-c3d7-40d3-88a8-d0d801917b7e/manager/1.log"
Oct 30 22:20:35.242962 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:35.242919    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5fb4c4694-qrm8j_7d0c6301-a15c-4879-92a3-973c6a5f5d94/maas-api/0.log"
Oct 30 22:20:40.417176 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:40.417149    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-lbxmv_6b85ec40-a08a-4220-b02a-9959b72251e1/discovery/0.log"
Oct 30 22:20:40.772587 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:40.772445    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c8944d49-9c745_05b9e5e9-fd34-46c1-9688-062a0c091b51/kube-auth-proxy/0.log"
Oct 30 22:20:41.476039 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:41.476012    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-6vwl6_065139b3-6b4a-450b-a2ec-d8a8cf01caf3/istio-proxy/0.log"
Oct 30 22:20:41.738191 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:41.738104    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-8b756_15ae86a1-5c75-45ff-b549-32aa463d30a7/istio-proxy/0.log"
Oct 30 22:20:43.265350 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:43.265314    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-xtfjt_9ad04b60-4d70-44de-badb-f602b63d1556/istio-proxy/0.log"
Oct 30 22:20:43.536959 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:43.536860    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-zr2km_773d8fc2-85de-47dd-81b5-6831f66f5594/istio-proxy/0.log"
Oct 30 22:20:44.030063 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:44.030024    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-6hstp_65d3c06a-089a-4595-a2fe-0c4a769b49aa/payload-pre-processing/0.log"
Oct 30 22:20:44.265099 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:44.265062    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:20:45.330808 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:45.330781    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-8687n_086aa9d4-d49b-4f59-a95f-2a6eb8a82928/main/0.log"
Oct 30 22:20:46.049939 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:46.049901    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8jjpqb_b5613d2c-edac-4610-9c85-03a98b6a8493/main/0.log"
Oct 30 22:20:46.823860 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:20:46.823829    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-7thfs_6523484b-480f-4c20-b123-8013f0ef711e/main/0.log"
Oct 30 22:20:52.355574 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:20:52.355531    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:21:20.815240 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:21:20.815181    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:21:21.035029 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:21:21.034998    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:21:22.354929 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:21:22.354888    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:21:52.355361 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:21:52.355273    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:22:22.354898 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:22:22.354864    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:22:52.354414 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:22:52.354375    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:22:57.956837 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:57.956806    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:22:58.178073 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.178042    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:22:58.550278 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550243    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pfnpp/must-gather-mjfhj"]
Oct 30 22:22:58.550741 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550725    5786 cpu_manager.go:414] "containerMap: removing container" podUID="953d11ed-8f69-4ce9-a81e-370ac067864a" containerName="payload-processing"
Oct 30 22:22:58.550791 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550744    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="953d11ed-8f69-4ce9-a81e-370ac067864a" containerName="payload-processing"
Oct 30 22:22:58.550791 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550758    5786 cpu_manager.go:414] "containerMap: removing container" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy"
Oct 30 22:22:58.550791 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550763    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy"
Oct 30 22:22:58.550892 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550792    5786 cpu_manager.go:414] "containerMap: removing container" podUID="9e587ffd-eb44-4774-a5d1-4d92d5cf71f8" containerName="payload-pre-processing"
Oct 30 22:22:58.550892 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550799    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e587ffd-eb44-4774-a5d1-4d92d5cf71f8" containerName="payload-pre-processing"
Oct 30 22:22:58.550960 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550893    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e587ffd-eb44-4774-a5d1-4d92d5cf71f8" containerName="payload-pre-processing"
Oct 30 22:22:58.550960 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550905    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="953d11ed-8f69-4ce9-a81e-370ac067864a" containerName="payload-processing"
Oct 30 22:22:58.550960 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550912    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy"
Oct 30 22:22:58.550960 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550920    5786 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy"
Oct 30 22:22:58.551085 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550994    5786 cpu_manager.go:414] "containerMap: removing container" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy"
Oct 30 22:22:58.551085 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.550999    5786 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f7d12d8-d801-4b9d-9d1a-cac6c39f442f" containerName="istio-proxy"
Oct 30 22:22:58.554034 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.554016    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pfnpp/must-gather-mjfhj"
Oct 30 22:22:58.613616 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.613567    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pfnpp\"/\"kube-root-ca.crt\""
Oct 30 22:22:58.655995 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.655956    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b10cccc5-1048-4b8e-8f08-057bd3942ead-must-gather-output\") pod \"must-gather-mjfhj\" (UID: \"b10cccc5-1048-4b8e-8f08-057bd3942ead\") " pod="openshift-must-gather-pfnpp/must-gather-mjfhj"
Oct 30 22:22:58.656148 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.656115    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kchxb\" (UniqueName: \"kubernetes.io/projected/b10cccc5-1048-4b8e-8f08-057bd3942ead-kube-api-access-kchxb\") pod \"must-gather-mjfhj\" (UID: \"b10cccc5-1048-4b8e-8f08-057bd3942ead\") " pod="openshift-must-gather-pfnpp/must-gather-mjfhj"
Oct 30 22:22:58.659565 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.659537    5786 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pfnpp\"/\"openshift-service-ca.crt\""
Oct 30 22:22:58.659759 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.659567    5786 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pfnpp\"/\"default-dockercfg-k28w6\""
Oct 30 22:22:58.661155 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.661134    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pfnpp/must-gather-mjfhj"]
Oct 30 22:22:58.756985 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.756952    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b10cccc5-1048-4b8e-8f08-057bd3942ead-must-gather-output\") pod \"must-gather-mjfhj\" (UID: \"b10cccc5-1048-4b8e-8f08-057bd3942ead\") " pod="openshift-must-gather-pfnpp/must-gather-mjfhj"
Oct 30 22:22:58.757194 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.757037    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kchxb\" (UniqueName: \"kubernetes.io/projected/b10cccc5-1048-4b8e-8f08-057bd3942ead-kube-api-access-kchxb\") pod \"must-gather-mjfhj\" (UID: \"b10cccc5-1048-4b8e-8f08-057bd3942ead\") " pod="openshift-must-gather-pfnpp/must-gather-mjfhj"
Oct 30 22:22:58.757378 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.757355    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b10cccc5-1048-4b8e-8f08-057bd3942ead-must-gather-output\") pod \"must-gather-mjfhj\" (UID: \"b10cccc5-1048-4b8e-8f08-057bd3942ead\") " pod="openshift-must-gather-pfnpp/must-gather-mjfhj"
Oct 30 22:22:58.814345 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.814275    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kchxb\" (UniqueName: \"kubernetes.io/projected/b10cccc5-1048-4b8e-8f08-057bd3942ead-kube-api-access-kchxb\") pod \"must-gather-mjfhj\" (UID: \"b10cccc5-1048-4b8e-8f08-057bd3942ead\") " pod="openshift-must-gather-pfnpp/must-gather-mjfhj"
Oct 30 22:22:58.868099 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:58.868063    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pfnpp/must-gather-mjfhj"
Oct 30 22:22:59.240438 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:59.240407    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pfnpp/must-gather-mjfhj"]
Oct 30 22:22:59.244096 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:22:59.244071    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb10cccc5_1048_4b8e_8f08_057bd3942ead.slice/crio-da25017a9088a35ba48892425cd15ef736b4db51387f2dacbe2beb0bf3e1ecc2 WatchSource:0}: Error finding container da25017a9088a35ba48892425cd15ef736b4db51387f2dacbe2beb0bf3e1ecc2: Status 404 returned error can't find the container with id da25017a9088a35ba48892425cd15ef736b4db51387f2dacbe2beb0bf3e1ecc2
Oct 30 22:22:59.246510 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:59.246493    5786 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:22:59.976134 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:22:59.976098    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pfnpp/must-gather-mjfhj" event={"ID":"b10cccc5-1048-4b8e-8f08-057bd3942ead","Type":"ContainerStarted","Data":"da25017a9088a35ba48892425cd15ef736b4db51387f2dacbe2beb0bf3e1ecc2"}
Oct 30 22:23:00.982695 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:00.982649    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pfnpp/must-gather-mjfhj" event={"ID":"b10cccc5-1048-4b8e-8f08-057bd3942ead","Type":"ContainerStarted","Data":"8a7b2abf81fb0c021d9414db00a707a1ce673ad00039e2462722343cbfcde192"}
Oct 30 22:23:00.983088 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:00.982708    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pfnpp/must-gather-mjfhj" event={"ID":"b10cccc5-1048-4b8e-8f08-057bd3942ead","Type":"ContainerStarted","Data":"919cb22354db7708919d69934f7b28e73a1c80e13efcbf027ab9db4d72ec02cd"}
Oct 30 22:23:01.098630 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:01.098557    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-pfnpp/must-gather-mjfhj" podStartSLOduration=1.738958932 podStartE2EDuration="3.098542618s" totalImagesPullingTime="1.359583686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:01.097335106 +0000 UTC m=+4236.968454238" watchObservedRunningTime="2026-10-30 22:23:01.098542618 +0000 UTC m=+4236.969661739"
Oct 30 22:23:08.686418 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:08.686387    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-r59mn_0492a73c-e41b-46c7-ad57-1f8220dfcf3a/global-pull-secret-syncer/0.log"
Oct 30 22:23:11.573915 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:11.573884    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zxr26_06846bf4-72b9-4348-b4ae-5108221dbd0b/konnectivity-agent/0.log"
Oct 30 22:23:11.972257 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:11.972225    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-134.us-west-1.compute.internal_d2a11b2240e0ab276a762b4e71b7c25e/haproxy/0.log"
Oct 30 22:23:22.374568 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:23:22.374527    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:23:32.415743 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:32.415712    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c86fb99-95a6-4edc-98e0-19895d6aa48a/alertmanager/0.log"
Oct 30 22:23:32.828352 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:32.828271    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c86fb99-95a6-4edc-98e0-19895d6aa48a/config-reloader/0.log"
Oct 30 22:23:33.242945 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:33.242911    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c86fb99-95a6-4edc-98e0-19895d6aa48a/kube-rbac-proxy-web/0.log"
Oct 30 22:23:33.666776 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:33.666734    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c86fb99-95a6-4edc-98e0-19895d6aa48a/kube-rbac-proxy/0.log"
Oct 30 22:23:34.084984 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:34.084837    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c86fb99-95a6-4edc-98e0-19895d6aa48a/kube-rbac-proxy-metric/0.log"
Oct 30 22:23:34.499285 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:34.499256    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c86fb99-95a6-4edc-98e0-19895d6aa48a/prom-label-proxy/0.log"
Oct 30 22:23:34.913950 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:34.913926    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c86fb99-95a6-4edc-98e0-19895d6aa48a/init-config-reloader/0.log"
Oct 30 22:23:35.760369 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:35.760330    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-x6plj_0eb207a7-1d34-4edf-8e0a-488b4363e175/kube-state-metrics/0.log"
Oct 30 22:23:36.168224 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:36.168174    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-x6plj_0eb207a7-1d34-4edf-8e0a-488b4363e175/kube-rbac-proxy-main/0.log"
Oct 30 22:23:36.584602 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:36.584493    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-x6plj_0eb207a7-1d34-4edf-8e0a-488b4363e175/kube-rbac-proxy-self/0.log"
Oct 30 22:23:37.003237 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.003182    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7c86794bcf-ck4wn_fdf5f020-799f-4fc6-86c2-27b76e7bb567/metrics-server/0.log"
Oct 30 22:23:37.278427 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.278287    5786 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"]
Oct 30 22:23:37.285009 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.284977    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.340176 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.340133    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"]
Oct 30 22:23:37.358825 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.358766    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-proc\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.359031 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.358907    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-podres\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.359138 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.359119    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-lib-modules\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.359246 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.359193    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-sys\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.359336 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.359246    5786 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfk6d\" (UniqueName: \"kubernetes.io/projected/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-kube-api-access-sfk6d\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.429607 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.429572    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-jp9t6_fe7b8332-a94f-40a0-b4b5-5880f10b571a/monitoring-plugin/0.log"
Oct 30 22:23:37.459927 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.459891    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-lib-modules\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.460173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.459960    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-sys\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.460173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.459982    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk6d\" (UniqueName: \"kubernetes.io/projected/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-kube-api-access-sfk6d\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.460173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.460010    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-proc\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.460173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.460054    5786 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-podres\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.460173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.460083    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-sys\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.460173 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.460100    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-lib-modules\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.460567 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.460178    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-podres\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.460567 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.460329    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-proc\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.524010 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.523973    5786 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfk6d\" (UniqueName: \"kubernetes.io/projected/3d36f1c9-ce3e-4265-b2fb-ca18283cb561-kube-api-access-sfk6d\") pod \"perf-node-gather-daemonset-6fj5r\" (UID: \"3d36f1c9-ce3e-4265-b2fb-ca18283cb561\") " pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.601686 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.601602    5786 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:37.880176 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.880101    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gr7z_9cf3b1a7-af00-46ce-a1fc-70b02516a53f/node-exporter/0.log"
Oct 30 22:23:37.990519 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:37.990480    5786 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"]
Oct 30 22:23:37.994232 ip-10-0-132-134 kubenswrapper[5786]: W1030 22:23:37.994175    5786 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3d36f1c9_ce3e_4265_b2fb_ca18283cb561.slice/crio-740f72ba67e83e740aa2fc50f58b6c82bb44b0690cd62c5295c8c664c7f32de9 WatchSource:0}: Error finding container 740f72ba67e83e740aa2fc50f58b6c82bb44b0690cd62c5295c8c664c7f32de9: Status 404 returned error can't find the container with id 740f72ba67e83e740aa2fc50f58b6c82bb44b0690cd62c5295c8c664c7f32de9
Oct 30 22:23:38.213918 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:38.213881    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r" event={"ID":"3d36f1c9-ce3e-4265-b2fb-ca18283cb561","Type":"ContainerStarted","Data":"8e89c28564863fef34810f00e0072d4c18c67de02e22575e4a55fe48974d0955"}
Oct 30 22:23:38.214143 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:38.213973    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r" event={"ID":"3d36f1c9-ce3e-4265-b2fb-ca18283cb561","Type":"ContainerStarted","Data":"740f72ba67e83e740aa2fc50f58b6c82bb44b0690cd62c5295c8c664c7f32de9"}
Oct 30 22:23:38.214339 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:38.214306    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:38.295172 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:38.295138    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gr7z_9cf3b1a7-af00-46ce-a1fc-70b02516a53f/kube-rbac-proxy/0.log"
Oct 30 22:23:38.298481 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:38.298457    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-lbxmv_6b85ec40-a08a-4220-b02a-9959b72251e1/discovery/0.log"
Oct 30 22:23:38.329675 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:38.329604    5786 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r" podStartSLOduration=1.329585566 podStartE2EDuration="1.329585566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:23:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:38.328373368 +0000 UTC m=+4274.199492508" watchObservedRunningTime="2026-10-30 22:23:38.329585566 +0000 UTC m=+4274.200704688"
Oct 30 22:23:38.405756 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:38.405726    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c8944d49-9c745_05b9e5e9-fd34-46c1-9688-062a0c091b51/kube-auth-proxy/0.log"
Oct 30 22:23:38.709799 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:38.709773    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gr7z_9cf3b1a7-af00-46ce-a1fc-70b02516a53f/init-textfile/0.log"
Oct 30 22:23:39.040421 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:39.040331    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-6vwl6_065139b3-6b4a-450b-a2ec-d8a8cf01caf3/istio-proxy/0.log"
Oct 30 22:23:39.254827 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:39.254797    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-8b756_15ae86a1-5c75-45ff-b549-32aa463d30a7/istio-proxy/0.log"
Oct 30 22:23:40.531679 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:40.531651    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-xtfjt_9ad04b60-4d70-44de-badb-f602b63d1556/istio-proxy/0.log"
Oct 30 22:23:40.757927 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:40.757898    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-zr2km_773d8fc2-85de-47dd-81b5-6831f66f5594/istio-proxy/0.log"
Oct 30 22:23:41.211954 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:41.211927    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-6hstp_65d3c06a-089a-4595-a2fe-0c4a769b49aa/payload-pre-processing/0.log"
Oct 30 22:23:41.445087 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:41.445056    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:23:41.725025 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:41.724991    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-8295p_92966fe2-fe0f-494f-9bc8-87000dd086c9/kube-rbac-proxy-main/0.log"
Oct 30 22:23:42.175045 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:42.175021    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-8295p_92966fe2-fe0f-494f-9bc8-87000dd086c9/kube-rbac-proxy-self/0.log"
Oct 30 22:23:42.600588 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:42.600509    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-8295p_92966fe2-fe0f-494f-9bc8-87000dd086c9/openshift-state-metrics/0.log"
Oct 30 22:23:44.229467 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:44.229437    5786 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pfnpp/perf-node-gather-daemonset-6fj5r"
Oct 30 22:23:45.935276 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:45.935243    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-6rvsr_55bed0f4-2cfe-4624-90b8-2f56b728d578/prometheus-operator/0.log"
Oct 30 22:23:46.349244 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:46.349143    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-6rvsr_55bed0f4-2cfe-4624-90b8-2f56b728d578/kube-rbac-proxy/0.log"
Oct 30 22:23:46.878591 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:46.878554    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-s47fz_7535ee53-f698-47f2-83fa-e0694379654a/prometheus-operator-admission-webhook/0.log"
Oct 30 22:23:47.396988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:47.396962    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78659679-f77rg_2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/telemeter-client/0.log"
Oct 30 22:23:47.812580 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:47.812554    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78659679-f77rg_2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/reload/0.log"
Oct 30 22:23:48.227918 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:48.227886    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78659679-f77rg_2c7d804a-c2a5-4923-8a9b-ed2803fdd02a/kube-rbac-proxy/0.log"
Oct 30 22:23:48.640600 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:48.640514    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-lbxmv_6b85ec40-a08a-4220-b02a-9959b72251e1/discovery/0.log"
Oct 30 22:23:48.649741 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:48.649712    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-646bb56695-d6qr8_e75fd1f7-4fef-489b-ab98-45bbaf194bdf/thanos-query/0.log"
Oct 30 22:23:48.799119 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:48.799095    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c8944d49-9c745_05b9e5e9-fd34-46c1-9688-062a0c091b51/kube-auth-proxy/0.log"
Oct 30 22:23:49.063624 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:49.063600    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-646bb56695-d6qr8_e75fd1f7-4fef-489b-ab98-45bbaf194bdf/kube-rbac-proxy-web/0.log"
Oct 30 22:23:49.433617 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:49.433591    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-6vwl6_065139b3-6b4a-450b-a2ec-d8a8cf01caf3/istio-proxy/0.log"
Oct 30 22:23:49.481423 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:49.481395    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-646bb56695-d6qr8_e75fd1f7-4fef-489b-ab98-45bbaf194bdf/kube-rbac-proxy/0.log"
Oct 30 22:23:49.644453 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:49.644428    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-8b756_15ae86a1-5c75-45ff-b549-32aa463d30a7/istio-proxy/0.log"
Oct 30 22:23:49.894942 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:49.894870    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-646bb56695-d6qr8_e75fd1f7-4fef-489b-ab98-45bbaf194bdf/prom-label-proxy/0.log"
Oct 30 22:23:50.315154 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:50.315125    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-646bb56695-d6qr8_e75fd1f7-4fef-489b-ab98-45bbaf194bdf/kube-rbac-proxy-rules/0.log"
Oct 30 22:23:50.736268 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:50.736240    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-646bb56695-d6qr8_e75fd1f7-4fef-489b-ab98-45bbaf194bdf/kube-rbac-proxy-metrics/0.log"
Oct 30 22:23:50.933842 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:50.933813    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-xtfjt_9ad04b60-4d70-44de-badb-f602b63d1556/istio-proxy/0.log"
Oct 30 22:23:51.155563 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:51.155479    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-zr2km_773d8fc2-85de-47dd-81b5-6831f66f5594/istio-proxy/0.log"
Oct 30 22:23:51.572100 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:51.572064    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-6hstp_65d3c06a-089a-4595-a2fe-0c4a769b49aa/payload-pre-processing/0.log"
Oct 30 22:23:51.781632 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:51.781600    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:23:52.355222 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:23:52.355172    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:23:58.738556 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:58.738528    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64c49d8448-l4drv_c435ad8a-42bd-4839-b96e-4320ca0d44d5/console/0.log"
Oct 30 22:23:59.168688 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:23:59.168660    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-b2mvs_1daa2700-153a-4fed-8434-669dd9b021bb/download-server/0.log"
Oct 30 22:24:08.886620 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:08.886587    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sjfxt_b8b7c30e-9edc-43f6-ba65-279d40a3267e/dns/0.log"
Oct 30 22:24:09.300652 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:09.300621    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sjfxt_b8b7c30e-9edc-43f6-ba65-279d40a3267e/kube-rbac-proxy/0.log"
Oct 30 22:24:10.553441 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:10.553412    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nw865_e23751d7-2d21-4805-8a90-fd6276de3545/dns-node-resolver/0.log"
Oct 30 22:24:14.689576 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:14.689497    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mwrs5_d1a59846-fdda-4740-ada7-13de6a939e93/node-ca/0.log"
Oct 30 22:24:21.871701 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:21.871670    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-lbxmv_6b85ec40-a08a-4220-b02a-9959b72251e1/discovery/0.log"
Oct 30 22:24:22.185678 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:22.185649    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c8944d49-9c745_05b9e5e9-fd34-46c1-9688-062a0c091b51/kube-auth-proxy/0.log"
Oct 30 22:24:22.354472 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:24:22.354434    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:24:23.452727 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:23.452688    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-6vwl6_065139b3-6b4a-450b-a2ec-d8a8cf01caf3/istio-proxy/0.log"
Oct 30 22:24:23.870703 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:23.870615    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-8b756_15ae86a1-5c75-45ff-b549-32aa463d30a7/istio-proxy/0.log"
Oct 30 22:24:26.386087 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:26.386062    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-xtfjt_9ad04b60-4d70-44de-badb-f602b63d1556/istio-proxy/0.log"
Oct 30 22:24:26.809680 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:26.809642    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-zr2km_773d8fc2-85de-47dd-81b5-6831f66f5594/istio-proxy/0.log"
Oct 30 22:24:27.643297 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:27.643267    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-6hstp_65d3c06a-089a-4595-a2fe-0c4a769b49aa/payload-pre-processing/0.log"
Oct 30 22:24:28.061189 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:28.061159    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-z25fv_f94dd074-f073-4f4d-81eb-72ba79a39d9a/payload-processing/0.log"
Oct 30 22:24:32.224739 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.224655    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fvn9q_2238e566-a39a-47ff-8f14-a27a404bc255/serve-healthcheck-canary/0.log"
Oct 30 22:24:32.429457 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.429427    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"]
Oct 30 22:24:32.429674 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.429651    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerName="istio-proxy" containerID="cri-o://3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e" gracePeriod=30
Oct 30 22:24:32.479330 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.479254    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"]
Oct 30 22:24:32.479484 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.479465    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerName="istio-proxy" containerID="cri-o://0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0" gracePeriod=30
Oct 30 22:24:32.480570 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.480545    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"]
Oct 30 22:24:32.480750 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.480723    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" containerName="istio-proxy" containerID="cri-o://ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1" gracePeriod=30
Oct 30 22:24:32.528787 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.528763    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"]
Oct 30 22:24:32.528977 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:32.528945    5786 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerName="istio-proxy" containerID="cri-o://f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d" gracePeriod=30
Oct 30 22:24:37.761385 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.761349    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 22:24:37.805021 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.804992    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 22:24:37.850386 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.850359    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 22:24:37.859465 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.859438    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 22:24:37.877694 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877661    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-data\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.877881 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877706    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-data\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.877881 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877766    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ad04b60-4d70-44de-badb-f602b63d1556-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad04b60-4d70-44de-badb-f602b63d1556-istiod-ca-cert\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.877881 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877812    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-kube-api-access-kwlj8\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-kube-api-access-kwlj8\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.877881 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877832    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-envoy\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.877881 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877862    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-socket\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.878096 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877896    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-kube-api-access-tftxl\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-kube-api-access-tftxl\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.878096 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877928    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-data" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.878096 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.877950    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-token\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-token\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.878384 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878346    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-data" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.878384 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878366    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-socket" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.878752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878421    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-certs\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.878752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878457    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/773d8fc2-85de-47dd-81b5-6831f66f5594-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/773d8fc2-85de-47dd-81b5-6831f66f5594-istiod-ca-cert\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.878752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878499    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9ad04b60-4d70-44de-badb-f602b63d1556-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad04b60-4d70-44de-badb-f602b63d1556-istio-podinfo\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.878752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878544    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ad04b60-4d70-44de-badb-f602b63d1556-istiod-ca-cert" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:37.878752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878559    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-certs\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.878752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878623    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-credential-socket\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.878752 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878657    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-istio-token\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-istio-token\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.879168 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.878819    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-certs" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.880395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.879323    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/773d8fc2-85de-47dd-81b5-6831f66f5594-istiod-ca-cert" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:37.880395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.879632    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-credential-socket" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.880395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.879791    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-envoy\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.880395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.879836    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-credential-socket\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.880395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.879873    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-podinfo\") pod \"773d8fc2-85de-47dd-81b5-6831f66f5594\" (UID: \"773d8fc2-85de-47dd-81b5-6831f66f5594\") "
Oct 30 22:24:37.880395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.879930    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-socket\") pod \"9ad04b60-4d70-44de-badb-f602b63d1556\" (UID: \"9ad04b60-4d70-44de-badb-f602b63d1556\") "
Oct 30 22:24:37.880395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880113    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-credential-socket" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.880395 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880122    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-certs" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880518    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880536    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/773d8fc2-85de-47dd-81b5-6831f66f5594-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880547    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880560    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880560    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-kube-api-access-kwlj8" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "kube-api-access-kwlj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880572    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880587    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880599    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880611    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad04b60-4d70-44de-badb-f602b63d1556-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880624    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.880988 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.880838    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-socket" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.881518 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.881000    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-envoy" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.881518 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.881032    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-kube-api-access-tftxl" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "kube-api-access-tftxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:37.881518 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.881161    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-token" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:37.881518 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.881281    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ad04b60-4d70-44de-badb-f602b63d1556-istio-podinfo" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:24:37.882271 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.882250    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-istio-token" pod "9ad04b60-4d70-44de-badb-f602b63d1556" (UID: "9ad04b60-4d70-44de-badb-f602b63d1556"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:37.883192 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.883167    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-podinfo" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:24:37.883192 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.883170    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-envoy" pod "773d8fc2-85de-47dd-81b5-6831f66f5594" (UID: "773d8fc2-85de-47dd-81b5-6831f66f5594"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.981624    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-certs\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.981713    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-certs\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.981762    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-kube-api-access-xldsg\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-kube-api-access-xldsg\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.981795    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-data\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.981836    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-podinfo\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.981870    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-credential-socket\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.981915    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-socket\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.981944    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-envoy\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982255    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-data\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982301    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-kube-api-access-bq44p\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-kube-api-access-bq44p\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982337    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-token\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-token\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982380    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-credential-socket\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982461    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istiod-ca-cert\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982489    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15ae86a1-5c75-45ff-b549-32aa463d30a7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ae86a1-5c75-45ff-b549-32aa463d30a7-istiod-ca-cert\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982519    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-socket\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982569    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-certs" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982576    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-envoy\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982568    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-certs" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982664    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-podinfo\") pod \"15ae86a1-5c75-45ff-b549-32aa463d30a7\" (UID: \"15ae86a1-5c75-45ff-b549-32aa463d30a7\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982714    5786 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-token\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-token\") pod \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\" (UID: \"065139b3-6b4a-450b-a2ec-d8a8cf01caf3\") "
Oct 30 22:24:37.983046 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.982735    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-data" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983120    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983153    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983168    5786 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-certs\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983181    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwlj8\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-kube-api-access-kwlj8\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983197    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad04b60-4d70-44de-badb-f602b63d1556-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983227    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983265    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tftxl\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-kube-api-access-tftxl\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983278    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983290    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad04b60-4d70-44de-badb-f602b63d1556-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983303    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ad04b60-4d70-44de-badb-f602b63d1556-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983334    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983366    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/773d8fc2-85de-47dd-81b5-6831f66f5594-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983850    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15ae86a1-5c75-45ff-b549-32aa463d30a7-istiod-ca-cert" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983914    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-socket" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.983940    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istiod-ca-cert" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.984223    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-credential-socket" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.984416    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-socket" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.984634    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-credential-socket" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.984826 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.984687    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-data" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.986536 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.986505    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-kube-api-access-xldsg" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "kube-api-access-xldsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:37.987409 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.987233    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-envoy" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:37.987409 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.987329    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-kube-api-access-bq44p" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "kube-api-access-bq44p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:37.987409 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.987369    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-podinfo" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:24:37.987824 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.987795    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-token" pod "065139b3-6b4a-450b-a2ec-d8a8cf01caf3" (UID: "065139b3-6b4a-450b-a2ec-d8a8cf01caf3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:37.988471 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.988450    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-token" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:37.988718 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.988694    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-podinfo" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:24:37.989803 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:37.989788    5786 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-envoy" pod "15ae86a1-5c75-45ff-b549-32aa463d30a7" (UID: "15ae86a1-5c75-45ff-b549-32aa463d30a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:38.083968 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.083939    5786 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-data\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.083969    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq44p\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-kube-api-access-bq44p\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.083995    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084004    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084013    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084021    5786 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ae86a1-5c75-45ff-b549-32aa463d30a7-istiod-ca-cert\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084033    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084046    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084054    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084062    5786 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-token\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084071    5786 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xldsg\" (UniqueName: \"kubernetes.io/projected/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-kube-api-access-xldsg\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084080    5786 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-istio-podinfo\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084088    5786 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-credential-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084097    5786 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/065139b3-6b4a-450b-a2ec-d8a8cf01caf3-workload-socket\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.084145 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.084105    5786 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ae86a1-5c75-45ff-b549-32aa463d30a7-istio-envoy\") on node \"ip-10-0-132-134.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:38.108541 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.108517    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zwzcq_6b61fec4-1b8d-449c-b251-b48c3306d351/kube-rbac-proxy/0.log"
Oct 30 22:24:38.518558 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.518523    5786 generic.go:356] "Generic (PLEG): container finished" podID="9ad04b60-4d70-44de-badb-f602b63d1556" containerID="ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1" exitCode=0
Oct 30 22:24:38.518775 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.518581    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"
Oct 30 22:24:38.518775 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.518609    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" event={"ID":"9ad04b60-4d70-44de-badb-f602b63d1556","Type":"ContainerDied","Data":"ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1"}
Oct 30 22:24:38.518775 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.518664    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt" event={"ID":"9ad04b60-4d70-44de-badb-f602b63d1556","Type":"ContainerDied","Data":"8f0ff34af0727484f4f17ad80f3fc2c33cd28d32b797b11de88ed023b91787d8"}
Oct 30 22:24:38.518775 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.518698    5786 scope.go:122] "RemoveContainer" containerID="ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1"
Oct 30 22:24:38.520156 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.520127    5786 generic.go:356] "Generic (PLEG): container finished" podID="15ae86a1-5c75-45ff-b549-32aa463d30a7" containerID="0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0" exitCode=0
Oct 30 22:24:38.520298 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.520180    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"
Oct 30 22:24:38.520298 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.520177    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" event={"ID":"15ae86a1-5c75-45ff-b549-32aa463d30a7","Type":"ContainerDied","Data":"0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0"}
Oct 30 22:24:38.520298 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.520285    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756" event={"ID":"15ae86a1-5c75-45ff-b549-32aa463d30a7","Type":"ContainerDied","Data":"4f047f7e96974f7d4ca9335ccd17ee279e8ff8996266c20a18f9de2d348fa01e"}
Oct 30 22:24:38.522265 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.522242    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zwzcq_6b61fec4-1b8d-449c-b251-b48c3306d351/exporter/0.log"
Oct 30 22:24:38.522375 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.522289    5786 generic.go:356] "Generic (PLEG): container finished" podID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" containerID="f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d" exitCode=0
Oct 30 22:24:38.522375 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.522322    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"
Oct 30 22:24:38.522375 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.522352    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" event={"ID":"065139b3-6b4a-450b-a2ec-d8a8cf01caf3","Type":"ContainerDied","Data":"f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d"}
Oct 30 22:24:38.522577 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.522390    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6" event={"ID":"065139b3-6b4a-450b-a2ec-d8a8cf01caf3","Type":"ContainerDied","Data":"e4b2812cc9fd98de24bfdc97b40b27615fc2aff275c385464c15e6027827b21c"}
Oct 30 22:24:38.523861 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.523840    5786 generic.go:356] "Generic (PLEG): container finished" podID="773d8fc2-85de-47dd-81b5-6831f66f5594" containerID="3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e" exitCode=0
Oct 30 22:24:38.523956 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.523886    5786 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"
Oct 30 22:24:38.523956 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.523915    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" event={"ID":"773d8fc2-85de-47dd-81b5-6831f66f5594","Type":"ContainerDied","Data":"3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e"}
Oct 30 22:24:38.523956 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.523950    5786 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km" event={"ID":"773d8fc2-85de-47dd-81b5-6831f66f5594","Type":"ContainerDied","Data":"5cb5db246f0c7ecfce9be467b84bbabe212d96661eab8969ef5770bf7f942ea2"}
Oct 30 22:24:38.529852 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.529827    5786 scope.go:122] "RemoveContainer" containerID="ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1"
Oct 30 22:24:38.530110 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:24:38.530092    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1\": container with ID starting with ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1 not found: ID does not exist" containerID="ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1"
Oct 30 22:24:38.530169 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.530118    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1"} err="failed to get container status \"ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1\": rpc error: code = NotFound desc = could not find container \"ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1\": container with ID starting with ed08569005f13fdb5dbaad4e927167e36fe3375a9ed1e41741f5af6c9a3e01c1 not found: ID does not exist"
Oct 30 22:24:38.530169 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.530133    5786 scope.go:122] "RemoveContainer" containerID="0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0"
Oct 30 22:24:38.538108 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.538092    5786 scope.go:122] "RemoveContainer" containerID="0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0"
Oct 30 22:24:38.538384 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:24:38.538365    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0\": container with ID starting with 0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0 not found: ID does not exist" containerID="0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0"
Oct 30 22:24:38.538490 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.538392    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0"} err="failed to get container status \"0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0\": rpc error: code = NotFound desc = could not find container \"0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0\": container with ID starting with 0f8546d9f1d7c3d7668d7a1b8cbab9088b264146df3366ef0ebc83317c3addf0 not found: ID does not exist"
Oct 30 22:24:38.538490 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.538408    5786 scope.go:122] "RemoveContainer" containerID="f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d"
Oct 30 22:24:38.546299 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.546283    5786 scope.go:122] "RemoveContainer" containerID="f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d"
Oct 30 22:24:38.546569 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:24:38.546548    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d\": container with ID starting with f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d not found: ID does not exist" containerID="f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d"
Oct 30 22:24:38.546662 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.546575    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d"} err="failed to get container status \"f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d\": rpc error: code = NotFound desc = could not find container \"f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d\": container with ID starting with f920814c7a2069e737b81148221b899fce4848fb652f06342d5a724d7f23181d not found: ID does not exist"
Oct 30 22:24:38.546662 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.546590    5786 scope.go:122] "RemoveContainer" containerID="3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e"
Oct 30 22:24:38.558068 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.558047    5786 scope.go:122] "RemoveContainer" containerID="3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e"
Oct 30 22:24:38.558465 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:24:38.558446    5786 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e\": container with ID starting with 3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e not found: ID does not exist" containerID="3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e"
Oct 30 22:24:38.558533 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.558475    5786 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e"} err="failed to get container status \"3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e\": rpc error: code = NotFound desc = could not find container \"3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e\": container with ID starting with 3bca8890cfb3b77848401c15c5043c3195eb397f051ebc5f1ef4ad2e7d76610e not found: ID does not exist"
Oct 30 22:24:38.680718 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.680683    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"]
Oct 30 22:24:38.732341 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.731990    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-zr2km"]
Oct 30 22:24:38.838580 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.838543    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"]
Oct 30 22:24:38.843934 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.843901    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-6vwl6"]
Oct 30 22:24:38.936603 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.936515    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zwzcq_6b61fec4-1b8d-449c-b251-b48c3306d351/extractor/0.log"
Oct 30 22:24:38.951237 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.951195    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"]
Oct 30 22:24:38.954526 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:38.954492    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8b756"]
Oct 30 22:24:39.062601 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:39.062560    5786 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"]
Oct 30 22:24:39.065925 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:39.065897    5786 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-xtfjt"]
Oct 30 22:24:40.358422 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:40.358370    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="065139b3-6b4a-450b-a2ec-d8a8cf01caf3" path="/var/lib/kubelet/pods/065139b3-6b4a-450b-a2ec-d8a8cf01caf3/volumes"
Oct 30 22:24:40.359045 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:40.359021    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15ae86a1-5c75-45ff-b549-32aa463d30a7" path="/var/lib/kubelet/pods/15ae86a1-5c75-45ff-b549-32aa463d30a7/volumes"
Oct 30 22:24:40.359513 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:40.359492    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="773d8fc2-85de-47dd-81b5-6831f66f5594" path="/var/lib/kubelet/pods/773d8fc2-85de-47dd-81b5-6831f66f5594/volumes"
Oct 30 22:24:40.360256 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:40.360237    5786 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ad04b60-4d70-44de-badb-f602b63d1556" path="/var/lib/kubelet/pods/9ad04b60-4d70-44de-badb-f602b63d1556/volumes"
Oct 30 22:24:44.902305 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:44.902277    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6c776cd74c-njc6b_8222c43e-fb37-4521-94fd-a4698e0876e4/manager/0.log"
Oct 30 22:24:45.316166 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:45.316139    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6c776cd74c-njc6b_8222c43e-fb37-4521-94fd-a4698e0876e4/copy-manifests/0.log"
Oct 30 22:24:49.280264 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:49.280230    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-78bd8688d6-29hbh_e06518d1-5603-4766-ba64-8d71103f8025/manager/0.log"
Oct 30 22:24:49.280668 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:49.280568    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-78bd8688d6-29hbh_e06518d1-5603-4766-ba64-8d71103f8025/manager/1.log"
Oct 30 22:24:52.090235 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:52.090183    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-76d5869bc7-svshz_ec5111f7-df66-489c-8fe3-95c2a03aa914/manager/0.log"
Oct 30 22:24:52.356584 ip-10-0-132-134 kubenswrapper[5786]: E1030 22:24:52.356496    5786 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwzcq" containerName="extractor"
Oct 30 22:24:55.185249 ip-10-0-132-134 kubenswrapper[5786]: I1030 22:24:55.185201    5786 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-7dx2p_98ff4001-c3d7-40d3-88a8-d0d801917b7e/manager/0.log"